cyber-dojo infrastructure makeover

https://cyber-dojo.org is getting an infrastructure makeover. We are switching from K8S to AWS. It will start Wednesday 31st August at 10:00 AM London time.

Update
We've had to postpone the switchover.
It will now start Wednesday 7th September at 10:00 AM London time.
The switch over has three parts:

• First, we will switch the DNS so https://cyber-dojo.org points to the new AWS environments. At this point, all existing katas (each with their own 6 digit ID) will not be reachable. Creation of new katas will still work.
• Second, we will make a copy of all the data from the old K8S volume-mounts. This may take a while to complete as we copy 50GB+ of data bit by bit.
• Third, we will copy this data into the new AWS volume. At this point all katas will be available again.

The basic advice is simple.

• Don't create a kata before the switchover time and expect it to work as you cross the switchover time.
• Start and complete a kata either before or after the switchover time.

cyber-dojo Interview

I was recently interviewed about the creation and use of cyber-dojo by Lancer Kind as part of his Agile Thoughts podcast series. All four episodes are now up.

• 193: Running coding dojos with cyber-dopo
• 194: Repating the same exercise, deep retrospectives, feedback loops
• 195: Teaching with cyber-dojo
• 196: Contribute to cyber-dojo

Why not give them a listen!?
Cheers
Jon

Help! Download your session as a git repo in a tgz file?

Note: This feature is now implemented.

An upgrade to the way cyber-dojo stores its backend data is now live. It now uses a git repo for each individual session. This is partly so I can provide a [download] button which will give you a tgz file of your current cyber-dojo session, in git format, with a git commit for each test event. This will allow people to upload their sessions to github for example. However, I cannot get the download working though the nginx reverse proxy. Can anyone help, or know someone who could? Update - the [download] button is now implemented.

If you'd like to set up your own env and try to help you'll need docker installed:

• get the saver repo
• its README.md file tells you how to build an image and run the tests
• when you build an image (after editing source) it finishes by printing commit-sha/image-tag information, eg
  

  echo CYBER_DOJO_SAVER_SHA=4a0ac7d08196d5c19eb2467e84a0e13f5c3931c4
  echo CYBER_DOJO_SAVER_TAG=4a0ac7d    
      

• get the nginx repo
• this has a build_test_publish.sh script which rebuilds the nginx image
• when you run it (after editing nginx source) it finishes by printing commit-sha/image-tag information, eg
  

  echo CYBER_DOJO_NGINX_SHA=5c55ca0c3b88d9ba3917014ac5be6442416f762d
  echo CYBER_DOJO_NGINX_TAG=5c55ca0
      

• get the web repo
• its demo.sh script will bring up all services; you'll then have a cyber-dojo running on localhost
• it uses sh/echo_versioner_env_vars.sh
• you need to add the echo statements from a saver rebuild after each saver commit
• you need to add the echo statements from a nginx rebuild after each nginx commit
• for example:
  

  echo_versioner_env_vars()
  {
  	docker run --rm cyberdojo/versioner:latest
  	
  	echo CYBER_DOJO_WEB_SHA="$(image_sha)"
  	echo CYBER_DOJO_WEB_TAG="$(image_tag)"
      
  	#...
  	echo CYBER_DOJO_SAVER_SHA=4a0ac7d08196d5c19eb2467e84a0e13f5c3931c4
  	echo CYBER_DOJO_SAVER_TAG=4a0ac7d    
  
  	echo CYBER_DOJO_NGINX_SHA=5c55ca0c3b88d9ba3917014ac5be6442416f762d
  	echo CYBER_DOJO_NGINX_TAG=5c55ca0
  }
      

Thanks

When and why tag Docker images to :latest ?

When I work on cyber-dojo I do quite a lot of testing. To state the obvious, when testing you want to test the software as it actually runs, as it's actually released, as it's actually deployed, etc. As obvious as this is, it can be surprising how often a distortion occurs.

cyber-dojo deploys each microservice in its own docker container. It would be faster to run tests locally, not inside a docker container, but that would be a distortion. So my development script does this:

1. build a new docker image
2. run a docker container from the image
3. wait till the container is ready
4. run tests from inside the container

Note step 1 does not volume mount the code and tests. That would be quicker but would be a distortion.

Now, step 1 in a workflow such as this commonly tags the docker image to :latest and, if an image is actually deployed, retags it to something unique (eg the first 7 characters of the git commit sha) and publishes that image. This is another small distortion. The published image now has a different tag to the one used in step 4's testing.

A development loop such as this can quickly build up a lot of stale image layers. When you clean up all those stale images you'll bust your image layer cache.

So my actual development script does this:

1. delete all images except :latest (a continuous approach to deleting stale image layers)
2. build a docker image tagged directly from the git commit sha
3. retag this image to :latest (to prevent step 1 from busting the cache next time around)
4. run a docker container from step 2)
5. wait till the container is ready/healthy
6. run tests against this container
7. if deploying, deploy the image from step 2)

I also need to publish the :latest image. People running their own cyber-dojo server need a way to upgrade. And an upgrade needs a consistent set of image tags. So step 2) builds an image not only tagged to the git commit sha, but which also includes the full 40 character git commit sha as an environment variable (SHA) inside the image. This allows:

IMAGE=cyberdojo/runner SHA=$(docker run --rm ${IMAGE}:latest bash -c 'echo ${SHA}') TAG=${SHA:0:7}

and the tagged image is ${IMAGE}:${TAG}

running your own cyber-dojo server

There are two ways to run your own cyber-dojo server:

• You can run on any Kubernetes cluster in a cloud or on a local minukube.
  Installation scripts and instructions are here.
• You can run on a plain Linux server with Docker installed.
  Instructions are here.

supporting the Turing Trust

We received this wonderful letter from James Turing of the Turing Trust.

To the cyber-dojo Foundation,

I wanted to share some positive news with you in these challenging times and an update on the important impact that your support of the Turing Trust is having. I am delighted to let you know that we have sent a further 1,600 PCs to Malawi. This is the largest shipment that we have ever sent to Malawi and it will go a long way to helping thousands more students access a digitally enabled education. Your donation enabled us to refurbish an additional 300 PCs that went into our recent shipment. Thank you again for your support that is fundamental to making our work possible as we get closer to realising our vision of a world where everyone has access to a technology-enabled education. Over the last few years, we have made tremendous progress, particularly in Malawi, where our projects have focused in the Northern Region of the country. Here we have increased the percentage of secondary schools that have computer labs from just 2% up to 32% by the end of 2019. You can see some of this progress on our interactive map showing the locations of these 63 schools as well as photos from each on our website. With your continuing support we believe that we can provide adequate computing facilities to every secondary school in Malawi by 2025.

We have also started a pilot programme working with primary schools. To date we have installed seven computer labs in primary schools. Each has 20 Mac Mini Computers running Linux Mint and loaded with educational resources using RACHEL. These computers have been received with great enthusiasm and are already having an impact on the way children are learning.

This initiative has made computer literacy a faster process whereby we can see children as young as four being able to name the parts of a computer and showing great mouse control. Nazir Quareshy, Headmaster for Bloom Junior Academy	It is a long unfulfilled dream achieved overnight. The board, teachers, members of staff and learners are all excited with the installation of computers at our school. They will enable us offer computer lessons but also be an important resource for teaching and learning. Keston Ndhlovu, School Adminstrator of Ekwendeni Church of Christ Primary School

In other good news, I am pleased to update you on our progress since we were accepted to be the first Growth Leader for the Raspberry Pi Foundation’s Code Club International by the Raspberry Pi CEO Philip Colligan. Our first Code Club has been established at Robert Laws Secondary School in the Mzimba District, Malawi. The club currently has 28 members who have completed courses in SCRATCH and HTML with a couple of students learning C++ as well. You can hear the students talking about their experiences in Code Club in this video.

Thank you so much for your support of The Turing Trust that is vital in enabling us to make such an impact.

Yours sincerely, James Turing

All this time at home, no commuting. I just want to code!

By Bob Allen.

If this sounds like your inner voice, you are not alone. So OK, let’s do the simplest thing that could possibly work, in the simplest way that could possibly work and keep our sanity.

Cyber-Dojo is the enabler I’ve been using for more than 6 years to help bring people together. Until recently that was by running a monthly coderetreat, where we choose any language plus any test framework we wish, and we pair-program with a different pair partner each hour. Participants are from all over the area, so it’s a great chance to meet and work side-by-side with people you’d otherwise never get the chance to code with.

Recent events, the pandemic, presented a new challenge; how to keep the community coming together virtually. Once again, cyber-dojo made this pretty much friction free. In late March, I got to mob program with people from all over the world for 2 hours at a time thanks to Woody Zuill. Last week, I hosted five days of two mobs per day, on three continents. The experience has been totally energizing and I’ve grown my network while having fun with friends both new and old.

Let’s Mob - Remote

Here’s how mobbing works when your sat together. Now, here’s how it can work when everyone is remote:

First, in what order the driver and navigator will rotate:

1. As a group (a mob) work out the sequence in which the driver role and navigator role will rotate. Record the participants names in one of the existing (or a new) documents on Cyber-Dojo. We often put this list into the feature file when we did an exercise using Cucumber, because it is where we tend to look whenever we changed to the next thing to do.
2. To determine sequencing for anything bigger than say 3-4 people, a trick Woody taught me goes like this: Using the chat feature of the video conferencing tool, ask everyone to enter the name of the city/country they are in. The order in which the names appear in the chat becomes your sequence for rotating the driver role. This has the added benefit of giving everyone some context about where their fellow mobbers are.
3. For a smaller mob, keep it simple; you can just enter people’s names and put that in the chat or in a document that everyone sees throughout the exercise.
4. The simplest way to rotate both driver and navigator is to have the current navigator becomes the next driver. This helps them to be focused when they become the driver.
5. Overriding the sequence for good: To help draw out someone who may be nervous about participating, make the least experienced member of the mob the first driver, and an experienced person be the first navigator. This has multiple benefits:
   1. The least experienced person gets the support they need from an experienced navigator (The rule in mobbing is: “For an idea to go from your head into the computer, it must go through someone else's hands.”
   2. Also, because when the least experienced person rotates off the driver’s seat, they now get the entire rotation to observe before it’s their turn to navigate.

How the driver change occurs:

Here are some options for limiting how long it takes to change drivers (changing drivers can be clumsy and interrupt the flow if it takes too much time):

1. Least cost: Don’t change the driver at all, i.e. no driver hand-off. Only rotate who is navigating.
2. Or, only change the driver at a rest break; fewer hand-off are less frequent,
3. Or, if possible, use the video conferencing app to transfer control to the driver. This way a single computer’s browser is controlling tests and the code, but, who is controlling the browser on that compute is what rotates. Note: For this option network latency can be difficult if some participants have not-so-good bandwidth or the mob is spread across long distances. In that case use the Hand-off-the-browser-protocol* to change drivers. Also, keyboard differences (Mac vs. Windows or Dvorak vs. QWERTY may be handled badly by the screen sharing software)
   *Hand-off-the-browser-protocol:
   After beginning an exercise, use the chat feature of your video sharing tool to give everyone the full URL to the Cyber-Dojo exercise. Make sure everyone who is going to be an active participant actually opens the URL in their browser. Make it clear that only the driver is allowed to hit the test button in the cyber-dojo upper left screen. This is important because that action “checks-in” whatever the state of all files currently on that individual’s browser. Now, when the time comes to switch drivers, make certain the current driven has checked-in any changes by hitting the test button. Only then can the next driver refresh their browser and be assured of being up-to-date. Have the new driver confirm the version displayed by Cyber-Dojo in their browser with the prior driver. In this way the hand-off can be confirmed.

Before you begin actually coding, the mob needs to make some choices:

1. If it’s just a single mob,
   • then from the opening page of cyber-dojo select “I’m on my own” followed by “create a new exercise”. You’ll be looking at a list of exercises/katas. There are just under 50 at present, each showing instructions on the right. Select one of them and hit next.
   • now choose a language and test framework, each showing freely given starting test code to the right. You may notice a hint of Douglas Adams in the tests regardless of the language or framework. Make your choice and away we go…
2. If you have too many people to stay as a single mob (rule of thumb: cap at 8-10 people*) you may want to use break-out rooms in your video conferencing software (Zoom does these well). In this case, each room becomes a separate mob and uses the practices described above in #1.
   *BIG mobs take a l-o-n-g time before the rotation comes back around and people’s attention tends to drift when this happens.

On what basis to change driver/navigator:

The usual practice is to use fixed timer (a simple count-down timer works fine if there is a designated time-keeper and they remember to start the timer. Forgetting to reset the timer is a common fail pattern).

Because remembering to restart the timer is common, we found another trigger for the rotation works for not-new mobs: getting to green. Upon the current navigator and driver completing a red-green cycle, initiate the change. Leave the choice to the next navigator to either refactor or to write another test to force new behavior. This boundary is a natural one because of the sense of completing a small change vs. an arbitrary time limit.

That said, if the mob finds itself doing big changes, including large refactoring, then use a timer with a short period like 5-6 minutes. This will help to influence the choice to make small changes: “The simplest thing that could possibly work”.

Here’s a little secret (an Easter egg?)

… not found in the “hotkeys” panel:

• Alt + mouse: Select column or rectangle of text. (Shift + up/down can also do this, but not in cyber-dojo)
• Ctrl + right/left: Seek beginning / end of next word. Hold Shift to select while seeking.

Typing, cut/paste, and seeks apply from current positions. These are the same as you will find in the JetBrains products.


Many thanks to Bob for this guest post.