cyber-dojo traffic-lights!

My friend Byran who works at the awesome Bluefruit Software in Redruth has hooked up his cyber-dojo web server to an actual traffic-light! Fantastic. Check out the video below :-)



Byran writes
It started out as a joke between myself and Josh (one of the testers at Bluefruit). I had the traffic lights in my office as I was preparing a stand to promote the outreach events (Summer Huddle, Mission to Mars, etc...) Software Cornwall runs. The conversation went on to alternative uses for the traffic lights, I was planning to see if people would pay attention to the traffic lights if I put them in a corridor at the event; we then came up with the idea that we could use them to indicate TDD test status.
Although it started out as a joke I am going to use it at the Summer Huddle, the lights change every time anyone runs a test so it should give an idea of how the entire group are doing without highlighting an individual pair.
The software setup is very simple, there is a Python web server (using the Flask library) running on a Raspberry Pi that controls the traffic lights using GPIO Zero. When the appendTestTrafficLight() function (in run_tests.js.erb) appends the traffic light image to the webpage I made it send an http 'get' request to the Raspberry Pi web server to set the physical traffic lights at the same time. At the moment the IP address of the Raspberry Pi is hard coded in the 'run_tests.js.erb' file so I have to rebuild the web image if anything changes but it was only meant to be a joke/proof of concept. The code is on a branch called traffic_lights on my fork of the cyber-dojo web repository.
The hardware is also relatively simple, there is a converter board on the Pi; this only converts the IO pin output connector of the Raspberry Pi to the cable that attaches to the traffic lights.
The other end of the cable from the converter board attaches to the board in the top left of the inside the traffic lights; this has some optoisolators that drive the relays in the top right which in turn switch on and off the transformers (the red thing in the bottom left) that drive the lights.
I have to give credit to Steve Amor for building the hardware for the traffic lights. They are usually used during events we run to teach coding to children (and sometimes adults). The converter board has LEDs, switches and buzzers on it to show that there isn't a difference between writing software to toggle LEDs vs driving actual real world systems, it's just what's attached to the pin. Having something where they can run the same code to drive LEDs and drive real traffic lights helps to emphasise this point.


a new architecture is coming

Between my recent fishing trips I have been working hard on a new cyber-dojo architecture.
  • pluggable start-points so you can now use your own language/tests/exercises lists on the setup page
  • a new setup page for custom start-points
  • once I've got the output parse functions inside the start-point volume I'll be switching the public cyber-dojo server to this image and updating the running-your-own-server instructions.
  • I've switched all development to a new github repo which has instructions if you want to try it now.


nordevcon cyber-dojo presentation

It was a pleasure to speak at the recent norfolk developers conference. My talk was "cyber-dojo: executing your code for fun and not for profit". I spoke about cyber-dojo, demo'd its features, discussed its history, design, difficulties and underlying technology. Videos of the talk are now on the infoq website. The slide-sync is not right at the start of part 2 but it soon gets corrected.

Docker tar pipe

This blog post is now out of date. The docker tar-pipe is now executed from micro-services implemented in Ruby.
For example, see runner.



I've been working on re-architecting cyber-dojo so the web-server (written in Rails) runs in a Docker image...
  • the server receives its source files from the browser
  • it saves them to a temporary folder
  • it back-ticks a shell file which
  • ...puts the source files into a docker container
  • ...runs the source files (by executing cyber-dojo.sh) as user=nobody
  • ...limits execution to 10 seconds


My shell file started like this:
#!/bin/sh SRC_DIR=$1 # where source files are IMAGE=$2 # the image to run them in MAX_SECS=$3 # how long they've got to complete TAR_FILE=`mktemp`.tgz # source files are tarred into this SANDBOX=/sandbox # where tar is untarred to inside container # - - - - - - - - - - - - - - - - - - - # 1. Create the tar file cd ${SRC_DIR} tar -zcf ${TAR_FILE} . # - - - - - - - - - - - - - - - - - - - # 2. Start the container CID=$(sudo docker run --detach \ --interactive \ --net=none \ --user=nobody \ ${IMAGE} sh) # - - - - - - - - - - - - - - - - - - - # 3. Pipe the source files into the container cat ${TAR_FILE} \ | sudo docker exec --interactive \ --user=root \ ${CID} \ sh -c "mkdir ${SANDBOX} \ && tar zxf - -C ${SANDBOX} \ && chown -R nobody ${SANDBOX}" # - - - - - - - - - - - - - - - - - - - # 4. After max_seconds, remove the container (sleep ${MAX_SECS} && sudo docker rm --force ${CID}) & # - - - - - - - - - - - - - - - - - - - # 5. Run cyber-dojo.sh in the container sudo docker exec --user=nobody \ ${CID} \ sh -c "cd ${SANDBOX} && ./cyber-dojo.sh 2>&1" # - - - - - - - - - - - - - - - - - - - # 6. If the container isn't running, the sleep woke and removed it RUNNING=$(sudo docker inspect --format="{{ .State.Running }}" ${CID}) if [ "${RUNNING}" != "true" ]; then exit 137 # (128=timed-out) + (9=killed) else exit 0 fi


Things to note:
  • The container is started in detached mode. This is so I can get its CID and setup the backgrounded sleep task (4) before running cyber-dojo.sh (5)
  • I use [sudo docker] because I do not put the current user into the docker group. Instead I sudo the current user to run the docker binary without a password.
  • The first [docker exec] user is root but this is root inside the CID container not root where the shell file is being run.
  • I can pipe STDIN from the shell into the container
  • The sleep task (4) kills the container if it runs out of time and step (6) detects this.


I realized I could avoid creating the (physical) tar file completely by using a 'proper' tar pipe:
#!/bin/sh ... (cd ${SRC_DIR} && tar -zcf - .) \ | sudo docker exec --interactive \ --user=root \ $CID \ sh -c "mkdir ${SANDBOX} \ && tar -zxf - -C ${SANDBOX} \ && chown -R nobody ${SANDBOX}" ...


  • [tar -zcf] means create a compressed tar file
  • [-] means don't write to a named file but to STDOUT
  • [.] means tar the current directory
  • which is why there's a preceding cd
At the other end of the pipe...
  • [tar -zxf] means extract files from the compressed tar file
  • [-] means don't read from a named file but from STDIN
  • [-C ${SANDBOX}] means save the extracted files to the ${SANDBOX} directory


Then I realized I could combine the two [docker exec]s into one and drop the chown...
... # - - - - - - - - - - - - - - - - - - - # 1. Start the container CID=$(sudo docker run \ --detach \ --interactive \ --net=none \ --user=nobody \ ${IMAGE} sh) # - - - - - - - - - - - - - - - - - - - # 2. After max_seconds, remove the container (sleep ${MAX_SECS} && sudo docker rm --force ${CID}) & # - - - - - - - - - - - - - - - - - - - # 3. Tar pipe the source files into the container and run (cd ${SRC_DIR} && tar -zcf - .) \ | sudo docker exec --interactive \ --user=nobody \ ${CID} \ sh -c "mkdir ${SANDBOX} \ && cd ${SANDBOX} \ && tar -zxf - -C . \ && ./cyber-dojo.sh" # - - - - - - - - - - - - - - - - - - - # 4. If the container isn't running, the sleep woke and removed it RUNNING=$(sudo docker inspect --format="{{ .State.Running }}" ${CID}) if [ "${RUNNING}" != "true" ]; then exit 137 # (128=timed-out) + (9=killed) else exit 0 fi


This worked but the backgrounded sleep created a zombie process.
That's for another blog.

Day of Docker keynote

It was an honour and a pleasure to open the Day of Docker conference in Oslo with a keynote telling the story of cyber-dojo (so far).