Docker for CI/CD

As our team has grown, we’ve seized the opportunity to improve processes that had previously been held back by our small size. Steering an organisation towards continuous delivery and away from the waterfall model is interesting in itself but for now I want to focus specifically on how well Docker is suited for CI/CD.

Docker recently (relatively) rolled out a tool called docker-compose. Very simply, it’s a YAML file that you can create yourself in under a minute that defines the set of Docker containers your application uses. For each container, you can either use your own Dockerfile or one on a public repository. You can also set any environment variables, links between containers and where your project should be mounted in each container.

For the developer, it’s remarkably simple but for the development process, it’s a paradigm shift. Now that docker-compose file has been created, it can be added to your Git/SVN repository so that not only is your code on the repository but the environment in which it runs is too.

This means that, if you deploy your application using the docker-compose file in your repository, in your staging environment, and everything works, then there is very little that would stop it working if you repeat the same process on production. It also means that, because the process is contained within a YAML file, if something does go wrong, not only do you have an audit trail for your code to help you but you also have one for the environment in which it runs.

This is the foundation of CD – but what about CI?

Well, if you use programmatic tests – be they unit tests, frontend test and/or server configuration tests, you can write a small script that runs them and add it to the repository as well. After setting up your containers with docker-compose, you can run your tests on the Docker containers it creates. If the tests pass on staging, it is very unlikely they would fail in production.

I can’t overstate just how easy this is and how well it can work. In just two months, we’ve moved all of our development to Docker-based CI/CD – and we’ve written our own software to automate all of the heavy lifting:

  1. We have a web hook set up in GitHub so that GitHub reports each push
  2. Our system (built in 2 days) runs clones the repository at the commit that was just pushed
  3. It then runs the docker-compose file contained within the repository and runs tests specified in a separate YAML file on the corresponding containers
  4. The output is captured and stored in the system and we get alerted via Slack if a test fails (typically we know if we’ve broken something within 3 minutes of pushing)
  5. If all of the tests pass and if the commits are to the master branch, another command could be run to copy the repository at that commit to production.



Leave a Reply

Your email address will not be published.