running your own cyber-dojo server

There are two ways to run your own cyber-dojo server:

You can run on any Kubernetes cluster in a cloud or on a local minukube.
Installation scripts and instructions are here.
You can run on a plain Linux server with Docker installed.
Instructions are here.

supporting the Turing Trust

We received this wonderful letter from James Turing of the Turing Trust.

I wanted to share some positive news with you in these challenging times and an update on the important impact that your support of the Turing Trust is having. I am delighted to let you know that we have sent a further 1,600 PCs to Malawi. This is the largest shipment that we have ever sent to Malawi and it will go a long way to helping thousands more students access a digitally enabled education.

Your donation enabled us to refurbish an additional 300 PCs that went into our recent shipment. Thank you again for your support that is fundamental to making our work possible as we get closer to realising our vision of a world where everyone has access to a technology-enabled education. Over the last few years, we have made tremendous progress, particularly in Malawi, where our projects have focused in the Northern Region of the country. Here we have increased the percentage of secondary schools that have computer labs from just 2% up to 32% by the end of 2019.

You can see some of this progress on our interactive map showing the locations of these 63 schools as well as photos from each on our website. With your continuing support we believe that we can provide adequate computing facilities to every secondary school in Malawi by 2025.

We have also started a pilot programme working with primary schools. To date we have installed seven computer labs in primary schools. Each has 20 Mac Mini Computers running Linux Mint and loaded with educational resources using RACHEL. These computers have been received with great enthusiasm and are already having an impact on the way children are learning.


This initiative has made computer literacy a faster process whereby we can see children as young as four being able to name the parts of a computer and showing great mouse control. Nazir Quareshy, Headmaster for Bloom Junior Academy	It is a long unfulfilled dream achieved overnight. The board, teachers, members of staff and learners are all excited with the installation of computers at our school. They will enable us offer computer lessons but also be an important resource for teaching and learning. Keston Ndhlovu, School Adminstrator of Ekwendeni Church of Christ Primary School

In other good news, I am pleased to update you on our progress since we were accepted to be the first Growth Leader for the Raspberry Pi Foundation’s Code Club International by the Raspberry Pi CEO Philip Colligan. Our first Code Club has been established at Robert Laws Secondary School in the Mzimba District, Malawi. The club currently has 28 members who have completed courses in SCRATCH and HTML with a couple of students learning C++ as well. You can hear the students talking about their experiences in Code Club in this video.

Thank you so much for your support of The Turing Trust that is vital in enabling us to make such an impact.

Yours sincerely, James Turing

All this time at home, no commuting. I just want to code!

By Bob Allen.

If this sounds like your inner voice, you are not alone. So OK, let’s do the simplest thing that could possibly work, in the simplest way that could possibly work and keep our sanity.

Cyber-Dojo is the enabler I’ve been using for more than 6 years to help bring people together. Until recently that was by running a monthly coderetreat, where we choose any language plus any test framework we wish, and we pair-program with a different pair partner each hour. Participants are from all over the area, so it’s a great chance to meet and work side-by-side with people you’d otherwise never get the chance to code with.

Recent events, the pandemic, presented a new challenge; how to keep the community coming together virtually. Once again, cyber-dojo made this pretty much friction free. In late March, I got to mob program with people from all over the world for 2 hours at a time thanks to Woody Zuill. Last week, I hosted five days of two mobs per day, on three continents. The experience has been totally energizing and I’ve grown my network while having fun with friends both new and old.

Let’s Mob - Remote

Here’s how mobbing works when your sat together. Now, here’s how it can work when everyone is remote:

First, in what order the driver and navigator will rotate:

As a group (a mob) work out the sequence in which the driver role and navigator role will rotate. Record the participants names in one of the existing (or a new) documents on Cyber-Dojo. We often put this list into the feature file when we did an exercise using Cucumber, because it is where we tend to look whenever we changed to the next thing to do.
To determine sequencing for anything bigger than say 3-4 people, a trick Woody taught me goes like this: Using the chat feature of the video conferencing tool, ask everyone to enter the name of the city/country they are in. The order in which the names appear in the chat becomes your sequence for rotating the driver role. This has the added benefit of giving everyone some context about where their fellow mobbers are.
For a smaller mob, keep it simple; you can just enter people’s names and put that in the chat or in a document that everyone sees throughout the exercise.
The simplest way to rotate both driver and navigator is to have the current navigator becomes the next driver. This helps them to be focused when they become the driver.
Overriding the sequence for good: To help draw out someone who may be nervous about participating, make the least experienced member of the mob the first driver, and an experienced person be the first navigator. This has multiple benefits:
1. The least experienced person gets the support they need from an experienced navigator (The rule in mobbing is: “For an idea to go from your head into the computer, it must go through someone else's hands.”
2. Also, because when the least experienced person rotates off the driver’s seat, they now get the entire rotation to observe before it’s their turn to navigate.

How the driver change occurs:

Here are some options for limiting how long it takes to change drivers (changing drivers can be clumsy and interrupt the flow if it takes too much time):

Least cost: Don’t change the driver at all, i.e. no driver hand-off. Only rotate who is navigating.
Or, only change the driver at a rest break; fewer hand-off are less frequent,
Or, if possible, use the video conferencing app to transfer control to the driver. This way a single computer’s browser is controlling tests and the code, but, who is controlling the browser on that compute is what rotates. Note: For this option network latency can be difficult if some participants have not-so-good bandwidth or the mob is spread across long distances. In that case use the Hand-off-the-browser-protocol* to change drivers. Also, keyboard differences (Mac vs. Windows or Dvorak vs. QWERTY may be handled badly by the screen sharing software)
*Hand-off-the-browser-protocol:
After beginning an exercise, use the chat feature of your video sharing tool to give everyone the full URL to the Cyber-Dojo exercise. Make sure everyone who is going to be an active participant actually opens the URL in their browser. Make it clear that only the driver is allowed to hit the test button in the cyber-dojo upper left screen. This is important because that action “checks-in” whatever the state of all files currently on that individual’s browser. Now, when the time comes to switch drivers, make certain the current driven has checked-in any changes by hitting the test button. Only then can the next driver refresh their browser and be assured of being up-to-date. Have the new driver confirm the version displayed by Cyber-Dojo in their browser with the prior driver. In this way the hand-off can be confirmed.

Before you begin actually coding, the mob needs to make some choices:

If it’s just a single mob,
- then from the opening page of cyber-dojo select “I’m on my own” followed by “create a new exercise”. You’ll be looking at a list of exercises/katas. There are just under 50 at present, each showing instructions on the right. Select one of them and hit next.
- now choose a language and test framework, each showing freely given starting test code to the right. You may notice a hint of Douglas Adams in the tests regardless of the language or framework. Make your choice and away we go…
If you have too many people to stay as a single mob (rule of thumb: cap at 8-10 people*) you may want to use break-out rooms in your video conferencing software (Zoom does these well). In this case, each room becomes a separate mob and uses the practices described above in #1.
*BIG mobs take a l-o-n-g time before the rotation comes back around and people’s attention tends to drift when this happens.

On what basis to change driver/navigator:

The usual practice is to use fixed timer (a simple count-down timer works fine if there is a designated time-keeper and they remember to start the timer. Forgetting to reset the timer is a common fail pattern).

Because remembering to restart the timer is common, we found another trigger for the rotation works for not-new mobs: getting to green. Upon the current navigator and driver completing a red-green cycle, initiate the change. Leave the choice to the next navigator to either refactor or to write another test to force new behavior. This boundary is a natural one because of the sense of completing a small change vs. an arbitrary time limit.

That said, if the mob finds itself doing big changes, including large refactoring, then use a timer with a short period like 5-6 minutes. This will help to influence the choice to make small changes: “The simplest thing that could possibly work”.

Here’s a little secret (an Easter egg?)

… not found in the “hotkeys” panel:

Alt + mouse: Select column or rectangle of text. (Shift + up/down can also do this, but not in cyber-dojo)
Ctrl + right/left: Seek beginning / end of next word. Hold Shift to select while seeking.

Typing, cut/paste, and seeks apply from current positions. These are the same as you will find in the JetBrains products.

Many thanks to Bob for this guest post.

enhanced diff/review

cyber-dojo has two new diff/review features.

The traffic-light navigator now has buttons to move to the first (left-most button) or last (right-most button) traffic-light.

Also, if you are in a group-exercise, there is an animal-navigator to move you to different animals.

Note that the [revert] button is now called the [checkout] button. This is because you can checkout (as in git checkout) the files from any traffic-light from any animal (rather than only reverting back to one of your own previous traffic-lights). If you're doing larger exercises this can be useful if you want to 'coverge' everyone to the same point.

Have fun!

themes + colour

cyber-dojo now supports light and dark themes as well as optional colour-syntax highlighting.

The [theme?] button toggles between the light and dark themes.
The [colour?] button toggles colour-syntax on/off.
The default is the dark-theme with colour-syntax on.

Click [theme?] to switch to light-theme (colour-syntax remains on).

Click [colour?] to switch colour-syntax off (theme remains light).

Click [theme?] to switch to dark-theme (colour-syntax remains off).

Have fun :-)

Securing Cyber-Dojo with SSL on Google Cloud

By Seb Rose. Cucumber has been running its own Cyber-Dojo instance for several years. Recently, due to changes to the cucumber website, it became necessary to support the HTTPS protocol. This turned out to be quite simple (once Jon fixed a defect we found in Cyber-Dojo’s nginx config), but I thought it might be useful to pull together the steps in one place.

[Big thanks to Jon Jagger, Mike Long, and Jon Skeet for their help getting us up and running]

Google Cloud VM

We operate our Cyber-Dojo instance in a virtual machine (VM) on Google Cloud (other cloud providers are available). While getting https set up, I created and deleted numerous VMs. Here I will outline the process I followed:

Go to console.cloud.google.com
Navigate to Compute Engine | VM Instances
Click Create Instance
Refer to Google’s help documentation for general guidance. Specific Cyber-Dojo advice is:
1. General Purpose Machine Family (N1 is fine)
2. Machine Type
  1. Custom
  2. 1 CPU
  3. 6GB memory
3. Boot disk
  1. OS image: Ubuntu 18.04¹
  2. Boot disk type: SSD persistent disk²
  3. Size: 100GB³
4. Firewall
  1. Allow http access
  2. Allow https access
Click Create and that’s the VM created

Install Cyber-Dojo

From the VM Instances page on Google Cloud, find the VM instance that you have just created and click the SSH link in the Connect column. This will open a terminal window.

Follow the instructions in this blog post to install Cyber-Dojo.

Part of these instructions require you to log out of the terminal and log back in again. This is important!

To log back in, simply click the SSH link again. All of the install instructions below need to be carried out in a terminal window.

Once you’ve installed Cyber-Dojo, follow the instructions to start it up and navigate to it using the External IP address shown in the VM Instance table. Note, you cannot click the IP address link to the left of the SSH link (in the Connect column) since that will prefix the IP address with https:// which is not yet installed. You must use the raw IP address. Cyber-Dojo should now be working correctly on http.

Now bring down your Cyber-Dojo server:

$ ./cyber-dojo down

Install nginx

Cyber-Dojo is made up of multiple services, each running in its own Docker container. One of the Docker containers uses nginx to process incoming requests. We could try to get this container to handle HTTPS requests, but that would require us to modify the Docker image. Mike Long suggested a simpler technique.

Mike’s suggestion was to install nginx directly on the VM. This instance of nginx will handle all incoming http and https requests, forwarding them over http to the Cyber-Dojo nginx container. This allows us to support https without making any modifications to any Docker image that ships with Cyber-Dojo.

On Ubuntu 18.04 follow the install instructions on the nginx website.

Now, if you navigate to the External IP address shown in the VM Instance table you should be presented with an nginx holding page, where before you would have seen Cyber-Dojo.

Install an SSL certificate

I used Certbot from the EFF to create and install an SSL certificate. The process is simple and well documented on their website. Follow the instructions to install Cerbot, but DO NOT RUN Certbot YET.

When you run Certbot, it will attempt to verify that the website is accessible via the domain name that certificate references. For this verification to be successful, you will need to ensure that your website URL points to the External IP address shown in the VM Instance table. How you do this depends on who provides your DNS services.

For our instance of Cyber-Dojo, we have a DNS A record for a sub-domain of cucumber.io (so the full domain name looked like, xxxxx.cucumber.io) which resolves to the External IP address shown in the VM Instance table.

Once you have setup your DNS, you can run Certbot from the terminal:

$ sudo certbot --nginx

You will be asked for several pieces of information during the process:

You need to provide the full domain name (not URL) for your website, eg ours looked like xxxxx.cucumber.io
Select the option to redirect all http requests to https

The whole process should take no longer than a minute and should complete without errors!

The certificate only lasts for 90 days, so you’ll probably want to set it to auto-renew. I created a cron job to do this every 2 months:

$ sudo crontab -e
Add the following text to the cron file: * * * */2 * certbot renew
Save the cron file

Configure nginx

To configure nginx, you’ll need to edit /etc/nginx/sites-available/default

The editors available by default are quite limited. I used nano.

$ cd /etc/nginx/sites-available $ sudo nano default

The configuration file (/etc/nginx/sites-available/default) has already been modified by Certbot - and you’ll see comments to that effect throughout it. The change you need to make is to use a location block inside the server block listening on port 443.

So, look for text that reads:

   listen [::]:443 ssl ipv6only=on; # managed by Certbot
   listen 443 ssl; # managed by Certbot

Slightly above these two lines is a short location / { … } block. Replace this location block with the following code:

          location / {
               proxy_pass         http://127.0.0.1:8000;
               proxy_redirect     off;
               proxy_set_header   Host             $host;
               proxy_set_header   X-Real-IP        $remote_addr;
               proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
               proxy_max_temp_file_size 0;
               client_max_body_size       10m;
               client_body_buffer_size    128k;
               proxy_connect_timeout      90;
               proxy_send_timeout         90;
               proxy_read_timeout         90;
               proxy_buffer_size          4k;
               proxy_buffers              4 32k;
               proxy_busy_buffers_size    64k;
               proxy_temp_file_write_size 64k;
           }

Check it:

$ sudo nginx -t

If it reports ok, restart the nginx server

$ sudo systemctl restart nginx

The final step

The most important statement in the location block is:

proxy_pass         http://127.0.0.1:8000;

This forwards all requests to port 8000, so Cyber-Dojo needs to be listening to requests on port 8000. This is very easy to do. Restart Cyber-Dojo using the following command:

$ ./cyber-dojo up --port=8000

Navigate to the URL that you specified in the DNS (e.g. xxxxx.cucumber.io) and you should now be able to use Cyber-Dojo over https :-)

Notes

I chose Ubuntu 18.04 because Cyber-Dojo has been tested on that platform. Other OS images may well work just as well. ↩
We use an SSD disk for speed - but this will cost you more than a standard persistent disk. ↩
100GB is possibly overkill - you may well be able to operate successfully with a much smaller drive. ↩

New start-point Architecture

There's been a major change to the cyber-dojo start-points architecture.
Start-points are now docker images rather than docker volumes.

Do an update

On your server, do an update:

$ ./cyber-dojo update

Bring up a server with named start-point images

When you bring up a cyber-dojo server, you can name start-point docker images.
For example, in this command:

$ ./cyber-dojo up \ --custom=acme/my-custom \ --exercises=acme/my-exercises \ --languages=acme/my-languages

the three start-point images are named acme/my-custom, acme/my-exercises, and acme/my-languages.
You can omit any of these three -- options and it will use a default.
The default --custom image name is cyberdojo/custom
The default --exercises image name is cyberdojo/exercises
The default --languages image name is cyberdojo/languages-common which holds start-points for the more common language+testFrameworks (C#, C++, Java, Javascript, Python).
These three default start-point images were created using, respectively:

$ ./cyber-dojo start-point create \ cyberdojo/custom \ --custom \ https://github.com/cyber-dojo/custom.git $ ./cyber-dojo start-point create \ cyberdojo/exercises \ --exercises \ https://github.com/cyber-dojo/exercises.git $ ./cyber-dojo start-point create \ cyberdojo/languages-common \ --languages \ $(curl --silent https://raw.githubusercontent.com/cyber-dojo/languages/master/url_list/common)

The image cyberdojo/languages-all holds start-points for all the language+testFrameworks. It was created using:

$ ./cyber-dojo start-point create \ cyberdojo/languages-all \ --languages \ $(curl --silent https://raw.githubusercontent.com/cyber-dojo/languages/master/url_list/all)

To see more help on the up command:

$ ./cyber-dojo up --help

Create your own start-point images

The --dir option was one way, in the old architecture, to create your own start-point volume. For example:

$ ./cyber-dojo start-point create \ tutorials \ --dir=/Users/fred/custom-tutorials

The --dir option is no longer supported. You must now explicitly name the start-point type (--custom, --exercises, --languages) and provide a list of git-repo URLs. For example, the command:

$ ./cyber-dojo start-point create \ tutorials \ --custom \ /Users/fred/custom-tutorials

now creates a --custom start-point image named tutorials and each URL must be git cloneable and contain at least one manifest.json file adhering to the start-point json format.

The --git option was another way, in the old architecure, to create your own start-point volume. For example:

$ ./cyber-dojo start-point create \ tdd \ --git=https://github.com/org/repo/exercises-tdd.git

The --git option is no longer supported. You must now explicitly name the start-point type (--custom, --exercises, --languages) and provide a list of git-repo URLs. For example, the command:

$ ./cyber-dojo start-point create \ tdd \ --exercises \ https://github.com/org/repo/exercises-tdd.git

now creates an --exercises start-point image named tdd and each URL must be git cloneable and contain at least one manifest.json file adhering to the start-point json format.

The --list option was the last way, in the old architecture, to create your own start-point volume. For example:

$ ./cyber-dojo start-point create \ ruby \ --list=https://raw.githubusercontent.com/org/repo/master/langs-ruby-urls

The --list option is no longer supported. You must now explicitly name the start-point type (--custom, --exercises, --languages) and provide a list of git-repo URLs. For example, the command:

$ ./cyber-dojo start-point create \ ruby \ --languages \ $(curl --silent https://raw.githubusercontent.com/org/repo/master/langs-ruby-urls)

now creates a --languages start-point image named ruby and the curl command must expand to a list of URLs each of which is git cloneable and contains at least one manifest.json file adhering to the start-point json format.

To see more help on the start-point create command:

$ ./cyber-dojo start-point create --help

Create a manifest.json file for each exercise

In the old architecture, each --exercises start-point entry was simply a file called instructions. The name associated with each instructions file (when you set up a practice session) was the name of the directory where it lived (with underscores replaced by spaces). In the new architecture, each --exercises start-point URL must be git cloneable and contain at least one manifest.json file adhering to a subset of the start-point json format:

You must specify only the display_name and visible_filenames entries.
visible_filenames cannot contain a file called cyber-dojo.sh

For example:

{ "display_name": "Fizz Buzz", "visible_filenames": [ "instructions" ] }

Inspect a start-point image

To show (in JSON format) each display_name (eg "C (gcc), assert"), git-repo URL+SHA and image_name.
For example:

$ docker pull cyberdojo/languages-common $ ./cyber-dojo start-point inspect cyberdojo/languages-common { ... "C#, NUnit": { "url": "https://github.com/cyber-dojo-languages/csharp-nunit", "sha": "97eb6f778b777ee7bc789a44417c3440cbe2439a", "image_name": "cyberdojofoundation/csharp_nunit" }, ... "C (gcc), assert": { "url": "https://github.com/cyber-dojo-languages/gcc-assert", "sha": "1a9724fde31295a08054d93695180d64fcf05ccd", "image_name": "cyberdojofoundation/gcc_assert" }, ... "Python, unittest": { "url": "https://github.com/cyber-dojo-languages/python-unittest", "sha": "843ebcf9e308ffb9bf686ef5e27e577f9395798b", "image_name": "cyberdojofoundation/python_unittest" }, ... }

List all start-point images

In JSON format, by type:

$ ./cyber-dojo start-point ls { "custom": [ "cyberdojo/custom:latest" ], "exercises": [ "cyberdojo/exercises:latest" ], "languages": [ "cyberdojo/languages-all:latest", "cyberdojo/languages-common:latest", "cyberdojo/languages-small:latest" ] }