BDDing your Infrastructure with Vagabond

Or how I learned to stop worrying and Love the virtualization

David Worth -


In a previously life... I was a Cowboy Sysadmin

We had loads of Xen VMs all managed by hand.

I was pretty good with the duct tape + SVN in order to keep things running

Found out about Puppet from @ckdake but it's hard and the team was very resistant.

Not-So-Secret Agenda

I also like beer.

... and was asked to do a tech-talk on Belgian Beer

... but I have to drive.

Better Background

Came to Highgroove and joined a crazy big analytics problem with a ton of AWS Infrastructure.

Lots of Chef and I was bad with it.

Even when I got better with it spinning up new hosts would not always work out of the box...

And never on a Saturday when I was alone...

"Testing" all of our recipes was

  1. vague as to how ...
  2. not an option.

and it made me worse/risk averse!


Delicious summer beer.

Saison Dupont

The gold standard in Saisons

What do you mean by BDD?

Probably not what you do...

I care that not only are my chef or puppet configurations "correct" but that they actually do the right thing...

So what is "the right thing" in this case?

After using a "devOps" platform (Chef, Puppet, etc.) on some servers we would want...

  • Packages are installed
  • The correct version of packages are installed.
  • Other packages that should be uninstalled actually are uninstalled...
  • Services that should be reachable are actually reachable...
  • Those services are actually running the correct versions!
  • Deployed applications are actually working!

Trappist Beers

Comparing them is hard because they vary so much.

So my favorite is Westmalle Tripel

The competition - a literature survey.

I'm not the only one to have felt this pain clearly.

Last year at ArrrrCamp Bernard Grymonpon gave a great talk on Testing Chef Recipes - Slidedeck

Foodcritic - a Linter which seems pretty cool but isn't a tester.. more of a style-guide. Seems like a Good Thing.

Rspec-chef - Doesn't appear super feature complete and last commit Jan 11, 2012

chefspec - "ChefSpec runs your cookbook but without actually converging the node that your examples are being executed on." Also seems cool but doesn't meet my needs!

To quote Bernard's talk "feels like testing chef, not the recipe"

minitest-chef-handler seems a lot like what I was shooting for!

Abbaye Ale - has nothing to do with the church really

Everyone knows Leffe (and it's GREAT) but have you tried...

Grimbergen Blonde


As usual, anything I do well someone else actually wrote and I enhance a tiny amount.


Originally written by @wfarr during his RailsMachine tenure. He even talked about it once.

It's a framework for creating new virtual machines with Vagrant and testing them!

Vagabond (con't)

Flemish Red

Sour and delicious. Drink them all.

In particular Rodenbach Grand Cru

but.. if you can find it on draught try Duchesse de Bourgogne

Vagabond (con't) - the additions

So... what did you do about it?

I checked off some of the TODO list (there are some hard problems in there! Particularly hard to do in a general way!)

Vagabond (con't) - File contents

Given a file the following is harder than it seems:

it { should contain "some content" } 

If you just want a string on a single line it's not so bad but could be super slow for big files.

You could implement Boyer-Moore...

(how has that not been added to StdLib if the Sieve of Eratosthenes has been?)

What about regexps? What about multiline regexps? On HUGE files?

Hard to do "right" (hard to define "right"!) Not so terrible if you are willing to do something and fail...


Super delicious strong beer...

Tripel Karmeliet

do not drink 7 of these in a night.

in NYC.

after warning your whole party how dangerous it is.


Remote Matchers e.g. "real" BDD

a portscan module to scan ports and services (via ruby-nmap) in order to determine what's actually running.

Belgian White

so it's hot but you don't want a Hefeweizen? Perfect!

Blanche de Chambly

(not technically Belgian.. Canadian but Belgian-style)

Remote Matcher (con't)

a "webapp" module

  • Actually wraps up a capybara-webkit instance
  • Write capybara specs and run them against a service that is deployed in the Vagrant instance!
describe "Testing Apache's default landing page", :remote => :true do
let(:default_landing_page_copy) { "It works!" }
  describe webapp(default_url) do
    it "should have a landing page" do
      visit "/"
      page.should have_content default_landing_page_copy


have you seen the glass?!?! It's also delicious.

From the same brouwerij as Tripel Karmeleit

Upgrade paths

So your infrastructure is on v1 of your cookbooks, and you want to upgrade to v2 and know that it works...

We can do that with git tags (though it's kinda Hacky/PoC right now)



Can't buy it in the states except in their special "builder pack". Go to the monastary to try it!

What needs to be done and can't be now?

~/some_project_with_vagrant$ cat Gemfile | grep vagabond
gem 'vagabond'

~/some_project_with_vagrant$ vagabond
# here all your tests verify what's in your Vagrantfile
~/some_project_with_vagrant_and_multiple_hosts_it_spins_up$ vagabond
# here all of your tests run _per_ Vagrant host spun up!
  # think a web-server and a separate database server


Blended Lambic (open/spontaneously fermented) beer. Very funky and VERY awesome

Drie Fonteinen - Oude Geuze

or if you're super daring try Doesjel

A final word on motivation:

Remember, if your specs are clean and concise you are not only testing but building an infrastructure specification for your servers!

Just like good tests for your applications.