When and why tag Docker images to :latest ?

When I work on cyber-dojo I do quite a lot of testing. To state the obvious, when testing you want to test the software as it actually runs, as it's actually released, as it's actually deployed, etc. As obvious as this is, it can be surprising how often a distortion occurs.

cyber-dojo deploys each microservice in its own docker container. It would be faster to run tests locally, not inside a docker container, but that would be a distortion. So my development script does this:

1. build a new docker image
2. run a docker container from the image
3. wait till the container is ready
4. run tests from inside the container

Note step 1 does not volume mount the code and tests. That would be quicker but would be a distortion.

Now, step 1 in a workflow such as this commonly tags the docker image to :latest and, if an image is actually deployed, retags it to something unique (eg the first 7 characters of the git commit sha) and publishes that image. This is another small distortion. The published image now has a different tag to the one used in step 4's testing.

A development loop such as this can quickly build up a lot of stale image layers. When you clean up all those stale images you'll bust your image layer cache.

So my actual development script does this:

1. delete all images except :latest (a continuous approach to deleting stale image layers)
2. build a docker image tagged directly from the git commit sha
3. retag this image to :latest (to prevent step 1 from busting the cache next time around)
4. run a docker container from step 2)
5. wait till the container is ready/healthy
6. run tests against this container
7. if deploying, deploy the image from step 2)

I also need to publish the :latest image. People running their own cyber-dojo server need a way to upgrade. And an upgrade needs a consistent set of image tags. So step 2) builds an image not only tagged to the git commit sha, but which also includes the full 40 character git commit sha as an environment variable (SHA) inside the image. This allows:

IMAGE=cyberdojo/runner SHA=$(docker run --rm ${IMAGE}:latest bash -c 'echo ${SHA}') TAG=${SHA:0:7}

and the tagged image is ${IMAGE}:${TAG}