Hello `cide`

cide is a fairly simple command-line tool that runs project tests in an isolated environment using Docker. I recently gave a talk on the subject. Here are the slides (video pending) but out of context they aren’t that useful so let me introduce to you why I created the tool and how it works.

The problem

We (at Pusher) are using Jenkins like so many organisations to continuously test our private repositories’ source code and help us quickly detect regressions. If we could we would love to use something like Travis-CI but Github’s permission model doesn’t allow us to do that.

In Jenkins, worker nodes checkout or code into a workspace and execute the jobs using the “jenkins” user. It means that project dependencies like the right version of ruby, libxml2 or GHC need to be installed on the machine. Each project, or even code branches, might have different and potentially conflicting dependencies. Jenkins allows you to tag worker boxes so potentially it’s possible to distribute different projects onto different boxes management of that might quickly start to become complex.

The solution

cide is a command-line tool that when executed reads a cide.yml file in the top of the project, builds a Dockerfile.cide file, builds an image and executes the selected script. All output is sent to stdout and success is determined by the script’s exit status.

In some way it copies the approach of Travis-CI but leaves the task scheduling up to another entity which in our case is Jenkins but it could work with BuildBot or others. Thanks to Docker each build is isolated which solves the dependencies issue nicely and worker boxes’ only dependency is cide.

Other benefits

Thanks to the Dockerfile layered approach to building files it’s possible to re-build dependencies only when they change. This speeds builds and saves some bandwidth.

One of the questions I got during the talk which I think it fair is: > Why use not a Dockerfile instead of the cide.yml file ? One of the requirements we had fairly quickly was to be able to pull multiple private repositories from github from within the project as dependencies. cide allows to inject a SSH key into the temporary image, something a Dockerfile wouldn’t be able to do. My hope is also that cide will gain language-detection features like Travis-CI which would remove most of the boilerplate that most of the Dockerfile have.

Because cide is just a command-line, the developers can invoke it on their own machine. This speeds things up quite a bit when trying to come up with the right cide.yml file or when trying to debug the project.

It is also possible to run temporary linked containers along the build. One of our projects runs MySQL for example and all the data is erased after the tests have been run.

Finally cide clean is a command that removes old containers, images and volumes and can be put in a cron task. The implementation was more complicated than expected and cide works around some of the quirks of Docker.

Getting started

cide depends on ruby 2.0+ and docker 1.5+ (and boot2docker on OSX). Run gem install cide to install the command-line tool.

In your project create a cide.yml file. The format is documented here and here is an example for a ruby project:

from: ruby:2.1
  # Works around GEM_HOME issues with the ruby:2.1 image
  - chown -R cide:cide /usr/local/bundle
    - Gemfile
    - Gemfile.lock
    - cide.gemspec
  run: bundle install --jobs=3 --retry=3 --deployment
run: bundle exec rake

Then run cide and the tests should start running.


That’s it really. Contact me or open an issue on the project if you want to discuss.