Travis-CI replacement with Jenkins and Docker. Sort of.

Hello world

./zimbatm.jpeg Jonas Pfenniger Chevalier aka. zimbatm developer and ops @ ./pusher-logo.png

Once upon a time

A little story in operations and development

Talking about how we ended-up building a replacement for Travis-CI with Jenkins and Docker.

Travis-CI

./TravisCI-Full-Color-vertical.png https://travis-ci.org

Service that runs tests.

First CI with deep GitHub integration.

Very simple to use.

Other competitors but this is the only Open-Source one.

Jenkins

./jenkins_logo.png https://jenkins-ci.org/

Formerly Hudson until Oracle inherited the project

Architecture: master agents job definitions job instances

Very modular. 1009 plugins O_O

Docker

./docker-logo.png https://www.docker.com/

Popular Linux container solution.

Used to run programs in isolation.

index -> image1+image2+image3 -> container

Dockerfile

git-commit-like layered disk volumes

secret plot

Context - April 2014

We use Travis-CI on most of our Open Source projects.

It’s working great for us.

Real world :/

No third-party access to private repos allowed !

Copycat

We want the same as Travis-CI for our private repos.

Jenkins + Docker !

Replacement checklist

Authentication with GitHub

Very simple project setup

./step-1.svg

Very simple build and test description (.travis.yml)

./step-2.svg

Pull-requests and `git push` trigger builds

./step-3.svg

Installing Jenkins

We use puppet

Jenkins is pretty hard to automate.

Some config trough UI other trough Puppet.

Thanks puppet-jenkins

https://github.com/jenkinsci/puppet-jenkins

Step 1

✓ Authentication with GitHub

github-oauth plugin

Very simple project setup

Very simple build and test description (.travis.yml)

Pull-requests and `git push` trigger builds

Step 2

✓ Authentication with GitHub

✓ Very simple project setup

Sort of. template-project plugin to copy a template project around.

Very simple build and test description (.travis.yml)

Pull-requests and `git push` trigger builds

Step 4

✓ Authentication with GitHub

✓ Very simple project setup

Very simple build and test description (.travis.yml)

✓ Pull-requests and `git push` trigger builds

github plugin. Needs a new GitHub user with admin access. Manages webhooks automatically.

more plugins !

embeddable-build-status

brings back the build badges

hipchat

for build notifications

gravatar

because github also uses gravatar

greenballs

because success is green.

timestamper

adds timestamps to the build logs. helps find where time is spent.

buildtriggerbadge

shows what causes a build to trigger.

Sort of

Dependency management

Where is step 3 ?

Simple YAML file to describe test build and run

Solving Step 3 - September 2014

Hello CIDE (Continuous Integration Docker Environment)

Weekend hack. 1 file, 145 lines of ruby.

gem install cide

Reads cide.yml

Builds temporary image with Docker

Runs tests in Docker

cide.yml example

---
from: ruby:2.1
as_root:
  - chown -R cide:cide /usr/local/bundle
before:
  add:
    - Gemfile
    - Gemfile.lock
    - cide.gemspec
  run: bundle install --jobs=3 --retry=3 --deployment
run: bundle exec rake

Command-line

$ cide --help
cide --help
Commands:
  cide build           # Builds an image and executes the run script
  cide clean           # Removes old containers
  cide debug           # Opens a debug console in the last project image
  cide help [COMMAND]  # Describe available commands or one specific command
  cide init            # Creates a blank cide.yml into the project

Runtime options:
  -f, [--force]                    # Overwrite files that already exist
  -p, [--pretend], [--no-pretend]  # Run but do not make any changes
  -q, [--quiet], [--no-quiet]      # Suppress status output
  -s, [--skip], [--no-skip]        # Skip files that already exist

More features

654 lines of code

Artifact exporting

SSH key injection

Linked containers

Demo

Sort of

Missing build matrix

Missing language detection

Linked containers might take a while to come up

Jenkins project per branch

Conclusion

Things are rarely as simple as they seem.

Pusher <3 Travis-CI

https://zimbatm.com

https://zimbatm.github.io/cide/

Thank you

:wq