Vagrant, Docker and Ansible. WTF?

Given that we’re building a SaaS that helps our client managing their infrastructure, our team is pretty familiar with leveraging VMs and configuration management tools. We’ve actually been heavy users of Vagrant and Ansible for the past year, and it’s helped us tremendously normalize our development process.

As our platform grew in complexity, some additional needs emerged:

• Containerization; we needed to be able to safely execute custom, and potentially harmful, code.
• Weight; as we added more sub-systems to devo.ps, having full blown VMs proved to be hard to juggle with when testing and developing.

And that’s why we ended up adding Docker to our development workflow. We were already familiar with it (as it powers some parts of the devo.ps infrastructure) and knew there would be obvious wins. In practice, we are shipping Docker containers in a main Vagrant image and drive some of the customization and upgrade with Ansible.

We’ll probably write something about this approach in the coming weeks, but given the amount of confusion there is around what these technologies are, and how they’re used, we thought we’d give you a quick tour on how to use them together.

Let’s get started.

Vagrant

You’ve probably heard about Vagrant; a healthy number of people have been writing about it in the past 6 months. For those of you who haven’t, think of it as a VM without the GUI. At its core, Vagrant is a simple wrapper around Virtualbox/VMware.

A few interesting features:

• Boatloads of existing images, just check Vagrantbox.es for example.
• Snapshot and package your current machine to a Vagrant box file (and, consequently, share it back).
• Ability to fine tune settings of the VM, including things like RAM, CPU, APIC…
• Vagrantfiles. This allows you to setup your box on init: installing packages, modifying configuration, moving code around…
• Integration with CM tools like Puppet, Chef and Ansible.

Let’s get it running on your machine:

1
2
3
4
5
6
7
8
9
10
11
12

1. First, [download Vagrant](http://downloads.vagrantup.com/) 
     and [VirtualBox](https://www.virtualbox.org/wiki/Downloads).
  2. Second, let's download an image, spin it up and SSH in:
   $ vagrant init precise64 http://files.vagrantup.com/precise64.box
   $ vagrant up
   $ vagrant ssh
  3. There's no 3.
  4. There's a 4 if you want to access your (soon to be) deployed app; 
  5. you will need to dig around the Vagrant documentation to 
     [perform port forwarding](http://docs.vagrantup.com/v2/networking/forwarded_ports.html), 
     [proper networking](http://docs.vagrantup.com/v2/networking/private_network.html) 
     and update manually your `Vagrantfile`.

Docker

Docker is a Linux container, written in Go (yay!) and based on lxc (self-described as “chroot on steroids”) and AUFS. Instead of providing a full VM, like you get with Vagrant, Docker provides you lightweight containers, that share the same kernel and allow to safely execute independent processes.

Docker is attractive for many reasons:

• Lightweight; images are much lighter than full VMs, and spinning off a new instance is lightning fast (in the range of seconds instead of minutes).
• Version control of the images, which makes it much more convenient to handle builds.
• Lots of images (again), just have a look at the docker public index of images.

Let’s set up a Docker container on your Vagrant machine:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73

1. SSH in Vagrant if you're not in already:
    
     $ vagrant ssh

  2. Install Docker, 
  [as explained on the official website](http://http://docs.docker.io/en/latest/installation/ubuntulinux/#ubuntu-precise-12-04-lts-64-bit):
    
     $ sudo apt-get update
     $ sudo apt-get install linux-image-generic-lts-raring linux-headers-generic-lts-raring
     $ sudo reboot
     $ sudo sh -c "curl https://get.docker.io/gpg | apt-key add -"
     $ sudo sh -c "echo deb http://get.docker.io/ubuntu docker main > /etc/apt/sources.list.d/docker.list"
     $ sudo apt-get update
     $ sudo apt-get install lxc-docker

  3. Verify it worked by trying to build your first container:
    
     $ sudo docker run -i -t ubuntu /bin/bash

  4. Great, but we'll need more than a vanilla Linux. To add our dependencies, for example to run a Node.js + MongoDB app, we're gonna start by creating a `Dockerfile`:
    
     FROM ubuntu
     MAINTAINER My Self me@example.com
     
     # Fetch Nodejs from the official repo (binary .. no hassle to build, etc.)
     ADD http://nodejs.org/dist/v0.10.19/node-v0.10.19-linux-x64.tar.gz /opt/
     
     # Untar and add to the PATH
     RUN cd /opt && tar xzf node-v0.10.19-linux-x64.tar.gz
     RUN ln -s /opt/node-v0.10.19-linux-x64 /opt/node
     RUN echo "export PATH=/opt/node/bin:$PATH" >> /etc/profile
     
     # A little cheat for upstart ;)
     RUN dpkg-divert --local --rename --add /sbin/initctl
     RUN ln -s /bin/true /sbin/initctl
     
     # Update apt sources list to fetch mongodb and a few key packages
     RUN echo "deb http://archive.ubuntu.com/ubuntu precise universe" >> /etc/apt/sources.list
     RUN apt-get update
     RUN apt-get install -y python git
     RUN apt-get install -y mongodb
     
     # Finally - we wanna be able to SSH in
     RUN apt-get install -y openssh-server
     RUN mkdir /var/run/sshd
     
     # And we want our SSH key to be added
     RUN mkdir /root/.ssh && chmod 700 /root/.ssh
     ADD id_rsa.pub /root/.ssh/authorized_keys
     RUN chmod 400 /root/.ssh/authorized_keys && chown root. /root/.ssh/authorized_keys
     
     # Expose a bunch of ports .. 22 for SSH and 3000 for our node app
     EXPOSE 22 3000
     
     ENTRYPOINT ["/usr/sbin/sshd", "-D"]

  5. Let's build our image now:
    
     $ sudo docker build .
     
     # Missing file id_rsa.pub ... hahaha ! You need an ssh key for your vagrant user
     $ ssh-keygen
     $ cp -a /home/vagrant/.ssh/id_rsa.pub .
     
     # Try again
     $ sudo docker build .
     
     # Great Success! High Five!

  6. Now, let's spin off a container with that setup and log into it (`$MY_NEW_IMAGE_ID` is the last id the build process returned to you):
    
     $ sudo docker run -p 40022:22 -p 80:3000 -d $MY_NEW_IMAGE_ID
     $ ssh root@localhost -p 40022

You now have a Docker container, inside a Vagrant box (Inception style), ready to run a Node.js app.

Ansible

Ansible is an orchestration and configuration management tool written in Python. If you want to learn more about Ansible (and you should…), we wrote about it a few weeks ago.

Let’s get to work. We’re now gonna deploy an app in our container:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62

1. [Install Ansible](/blog/2013/07/03/ansible-simply-kicks-ass.html), as we showed you in our previous post.

  2. Prepare your inventory file (`host`):
    
     app ansible_ssh_host=127.0.0.1 ansible_ssh_port=40022

  3. Create a simple playbook to deploy our app (`deploy.yml`):
    
     ---
     - hosts: app
       user: root
       tasks:
         # Fetch the code from github
         - name: Ensure we got the App code
           git:
             repo=git://github.com/madhums/node-express-mongoose-demo.git
             dest=/opt/node-express-mongoose-demo
         
         # NPM may or may not succeed, if you give it time, care, etc. it eventually works
         - name: Ensure the npm dependencies are installed
           command:
             chdir=/opt/node-express-mongoose-demo
             /opt/node/bin/npm install
           ignore_errors: yes
           
         # We will assume no changes in the default sample - or we should consider templates instead
         - name: Ensure the config files of the app
           command:
             creates=/opt/node-express-mongoose-demo/config/$item.js
             cp /opt/node-express-mongoose-demo/config/$item.example.js /opt/node-express-mongoose-demo/config/$item.js
           with_items:
             - config
             - imager
             
         # `initctl` is now linking to `true` and we have no access to services
         # Need to fake the start
         - name: Ensure mongodb data folders
           file:
             state=directory
             dest=$item
             owner=mongodb
             group=mongodb
           with_items:
             - /var/lib/mongodb
             - /var/log/mongodb
             
         # Super cheat combo !
         - name: Ensure mongodb is running
           shell:
             LC_ALL='C' /sbin/start-stop-daemon --background --start --quiet --chuid mongodb --exec  /usr/bin/mongod -- --config /etc/mongodb.conf
         
         # Cheating some more !
         - name: Ensure the App is running
           shell:
             chdir=/opt/node-express-mongoose-demo
             /opt/node/bin/npm start &

  4. Run that baby:
    
     $ ansible-playbook -i host deploy.yml

  5. We're done, point your browser at `http://localhost:80` - assuming you have performed the redirection mentioned in the initial setup of your vagrant box.

That’s it. You’ve just deployed your app on Docker (in Vagrant).

Let’s wrap it up

So we just saw (roughly) how these tools can be used, and how they can be complementary:

1. Vagrant will provide you with a full VM, including the OS. It’s great at providing you a Linux environment for example when you’re on MacOS.
2. Docker is a lightweight VM of some sort. It will allow you to build contained architectures faster and cheaper than with Vagrant.
3. Ansible is what you’ll use to orchestrate and fine-tune things. That’s what you want to structure your deployment and orchestration strategy.

It takes a bit of reading to get more familiar with these tools, and we’ll likely follow up on this post in the next few weeks. However, especially as a small team, this kind of technology allows you to automate and commoditize huge parts of your development and ops workflows. We strongly encourage you to make that investment. It has helped us tremendously increase the pace and quality of our throughput.

If you don’t know what librarian is – get to know it :) (it makes life much easier)[http://rubydoc.info/gems/librarian-chef] than way back when …

The most annoying thing happend to me earlier this evening, I was working on a cookbook [A chef cookbook …, I don’t actually do the cooking at home :)], and running librarian install yielded the following error:

"Cheffile and Cheffile.lock are out of sync!"

So like a good boy I remoced the Cheffile.lock and ran it again resulting with the same damn error message … arrrrrr !

So after examening my Cheffile over and over I noticed I have listed a dependency from two different location somthing like the followin [ clieng url’s omitted from snippet ]:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

 #!/usr/bin/env ruby
  #^syntax detection
  site "http://shelf.clients_domain.corp:8080"
  cookbook 'simple_proxy', :git => 'git://clients_domain.corp/hagzag/simple_proxy.git', :ref => 'master'
  cookbook 'nw-abap',     :git => 'git://clients_domain.corp/hagzag/nw-abap.git', :ref => 'master'
  cookbook 'sapinst'

  # for lvm volume creation do this once ...
  cookbook 'lvm', :site => 'http://clients_domain.corp:8080'
  cookbook 'sap-lvm'
  cookbook 'line',      :site => 'http://community.opscode.com/api/v1'

  cookbook 'nfs', :site => 'http://clients_domain.corp:8080'
  cookbook 'line', :site => 'http://clients_domain.corp:8080'

If you look a line numbers 11 & 14 you will see I am specifying the smae cookbook from tow different sources …, this is a good thing, your dependecny mechanisem / tool isn’t making decitions for you [ unlike Maven for exmaple … ].

So in order to avoid this error and to comtinue working I had to remove one if the lines and I was good to go.

1
2
3
4
5

 ...
  #cookbook 'line',      :site => 'http://community.opscode.com/api/v1'

  cookbook 'nfs', :site => 'http://clients_domain.corp:8080'
  cookbook 'line', :site => 'http://clients_domain.corp:8080'

Another fraction on my time line …

Enjoy.

Well just got a mac from my new project FB Status… – couldn’t resist it :)

One of the tasks was to setup an environment, and learn what a mac is :), a few days later I’m in love but still I need a way to destroy an recover my environment. I have a few plans like bash_it / oh_my_zsh & others, but until then I put togter a few scripts [ more to come ], the first of them is this one installing Xcode CLI Tools on Mac OSX Mountain Lion.

Install or (un-install) with the following simple command:

1
2

 sudo bash < <(curl -L https://raw.github.com/hagzag/xcode-cli-install/master/install.sh)
  sudo bash < <(curl -L https://raw.github.com/hagzag/xcode-cli-install/master/uninstall.sh)

The essence … see the rest on Github

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

detect_osx_version() {
  result=`sw_vers -productVersion`
  ...
  osxversion="10.8"
  osxvername="Mountain Lion"
  cltools=xcode462_cltools_10_86938259a.dmg
  mountpath="/Volumes/Command Line Tools (Mountain Lion)"
  mpkg="Command Line Tools (Mountain Lion).mpkg"
  pkg_url="https://www.dropbox.com/s/hw45wvjxrkrl59x/xcode462_cltools_10_86938259a.dmg"
  pkgmd5="90c5db99a589c269efa542ff0272fc28"
  ...
}
download_tools () {
  if [ -f /tmp/$cltools ]; then
    if [ `md5 -q /tmp/$cltools` = "${pkgmd5}" ]; then
      echo -e "$info $cltools already downloaded to /tmp/$cltools."
    else
       rm -f /tmp/$cltools
    fi
  else
    cd /tmp && wget $pkg_url -O ./$cltools
  fi
}

install_tools() {
  # Mount the Command Line Tools dmg
  echo -e "$info Mounting Command Line Tools..."
  hdiutil mount -nobrowse /tmp/$cltools
  # Run the Command Line Tools Installer
  echo -e "$info Installing Command Line Tools..."
  installer -pkg "$mountpath/$mpkg" -target "/Volumes/Macintosh HD"
  # Unmount the Command Line Tools dmg
  echo -e "$info Unmounting Command Line Tools..."
  hdiutil unmount "$mountpath"

  gcc_bin=`which gcc`
  gcc --version &>/dev/null && echo -e "$info gcc found in $gcc_bin"
}

I took the liberty (not sure it’s mine to take … but still), locating Xcode CLI tools for both mountain lion (10.8) and Lion (10.7) on DropBox, ecause Apple (like Oracle and other Giants) won’t let you download without a userid / pass which is quite irretating …

As Always hope you guy’s enjoy this. HP

Well a common use case for doing stuff with Jenkins jobs / configuration [ for example: enabling jobs, disabling jobs, executing jobs etc] is to use Jenkins-cli.jar which is included in your jenkins distribution.

In a large company where branches are branched on a daily basis and you need a build for each branch, job creation could become a bottleneck [ so could branch creation – but that a different topic :) ].

So my use case was to create a Job Generator which would take an existing job from branch 1.0 and create a job ( & the ranch but as I said different topic ) for branch 2.0 with the corresponding name, so as I mentioned thank god for Jenkins in general, Jenkins-cli and in addition to that a shell script, sed, xmlstarlet and again jenkins job parameters.

I needed a script which would export the job configuration to an xml file, search and replace for the SCM url, create the job so lets see how its done:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

execJenkinsCli(){ # excute jenkins-cli jar 
        jenkins_clinet_jar=./jenkins-cli.jar
        # download only if it's newwer than the local version [ curl -z ], the cli jar should be ofthe smae jenkins version ...
        curl -s -z ./jenkins-cli.jar ${JENKINS_URL}/jnlpJars/jenkins-cli.jar -o ${jenkins_clinet_jar} || { echo "Echo  cannot obtain jenkins-cli.jar exiting"; exit 2;}
        cli_cmd=$1
        cmd_params=$2
        cmd_prefix="${JAVA_CMD} -jar ${jenkins_clinet_jar} -s ${JENKINS_URL}"

        ${cmd_prefix} who-am-i | grep 'Authenticated as: anonymous' &>/dev/null && anon=0
        #anon=$?
        if [ "$anon" != "0" ]; then
                #printf "[ $FUNCNAME ] using jenkins-cli with private key auth"
                exec ${cmd_prefix} ${cli_cmd} ${cmd_params}
        elif [ "x" = "x${JENKINS_USER}" ] || [ "x"  = "x${JENKINS_PASS}" ]; then
                printf "[ $FUNCNAME ] cannot determin jenkins credentials\n";
                exit 3;
        else
                exec ${cmd_prefix} ${cli_cmd} ${cmd_params} --username ${JENKINS_USER} --password ${JENKINS_PASS}
        fi
}

execJenkinsCli get-job ${tmpl_job_name}  | \
  ${XML} ed -O -S -P -u '//project/scm/locations/hudson.scm.SubversionSCM_-ModuleLocation/remote' -v ${scm_url} | \
  execJenkinsCli create-job ${new_job_name}

  execJenkinsCli enable-job ${new_job_name} # our template is usually disabled, thus the cloned one is too ...

The full jobGen wrapper script can be found here

At DevOps Days Austin @mattray did an Openspace session on Omnibus which is a toolset based around the concept of installing an app and all of it’s prerequisites from source into a directory and then building a package ( either .deb or .rpm ) of that using fpm.

Having battled many times with OS Packages trying to get newer versions of Ruby, or Redis or other software installed and having to hunt down some random package repo or manually build from source this seems like an excellent idea.

To learn the basics I decided to build an omnibus package for fpm which helped me work out the kinks and learn the basics.

I found this information very supporting during my personal jorney into learning git, the internals IMO shows the mind shift one has to do when using Git after using any other SCM tool.

Setting up git over http (Smart HTTP in perticular) has it's caveats I will try to emphesize them as I go along.

Final Goal is to run git over http using git-http-backend (a.k.a smart http) in apposed to using webdav which is also called the dumb-method, webdav was never designed to work with git for the reason that it cannot compress object whilst transporting data between the client and the server hosting git [ there is no real git client – git server so i am not referring to it a such].

Versions used in this setup:

• Ubutnu version 12.04-LTS
• Git version 1.7.9.5
• Apache2 version 2.2.22

1. Install prequirests:

~4 years ago I wrote a script which will sync my drupal files directory between two servers [ I used rsync for that ], the problem with rsync was that it fetches a checsum of every file and direcroty over SSH which was very slow, and the critical issue was I have no history of the data – and if I am maintaining multipule syncs whats the point in rsync.

I looked @ svn at the time which was extreemly slow with binaries [ zip, tars etc ], Git and the git protocol in specific did the full sync [ ~1GB ] in under 45 minuets – over the Internet which was quite good IMO.

I was talking with a collegue on such a solution the other day and I thought why not share this. So for purely educational puposes I put thist scripup as a Gist @: https://gist.github.com/hagzag/5396510 [ or part of the repository @: https://github.com/hagzag/files_repo/blob/master/syncFiles.sh, but i am not sure it will survive there]  and an acompenying repository called "files_repo".

As you will see in the readme I am not sure it will do much for you but if you are consdering on using it read the following:

• create a repository [ e.g. on GitHub ]
• set the FILES_DIR parameter [ or default to /opt/data ]
• set the MY_GIT_REPO parameter [ or default to /opt/git-repo ]
• set my GIT_REMOTE parameter with the full git url of you repository [I didn't check this via https … ]

Run the script, this is really custom to my use case, but shows another awsome aspect of git.

Originally posted @: my Blog on tikalk.com

Lately I’ve been doing a lot of prototyping with Vagrant, specifically for a couple of distinct activities:–

• building puppet modules using the excellent puppet sandbox project
• and building RPM packages with FPM.

I realized I was spending a bunch of time flipping back and forth between Vagrant environments and I had no quick way to utilize RPMs built with FPM inside my puppet modules.

Problem

can’t install mysql2 gem :( See the full error below … (or in “Read on” link) what solved the issue was: installing the two missing dependencies “mysql-client libmysqlclient-dev”

1

sudo apt-get install mysql-client libmysqlclient-dev