Dustin Collins

DEVOPS ET CETERA

Read this first

Infrastructure with Python

Like Python? Into DevOps?

Here are some tools I use make my life a little easier.

awscli

THE command line interface if you’re using AWS. Really nice documentation on how to talk to the different AWS services. I use this a lot as a glue library. For example, once Jenkins runs tests on a project, we tar.gz an artifact and use awscli to upload it to S3.

boto

If you’re using AWS and need to get state or resources at runtime boto is what you want. The API is large, but well documented and composed.

botocore is a smaller low-level alternative, but I don’t care much about size when boto’s docs are good. botocore is the foundation for awscli.

Fabric

Fabric is my go-to tool for remote execution. PyChef fetches the hosts I need and passes them to Fabric, which takes care of running any commands. This is a popular lib that has been around for a while; a lot of...

Continue reading →


DevOps Resources

There are a lot of people on the web talking about every aspect of the DevOps movement, but it can be hard to find them. I’ve added some resources I use to a Github repo here. If you use any resources not on this list, feel free to open a pull request and we can build a great list of resources together.

Github: dustinmm80/devops_resources

My starter list:

Podcasts

Arrested DevOps

DevOps Cafe

Puppet Labs Podcast

The Cloudcast

The Food Fight Show

The Ship Show

Aggregators/Q&A

Dzone DevOps zone

dev2ops

devops.com

DevOps on Quora

LinkedIn DevOps group

/r/devops

Blogs (Orgs)

codeascraft (etsy)

Opscode Chef

Puppet

Blogs (Personal)

Andrew Clay Shafer

Bryan Berry

Chad Fowler

Joe Hirn

John Allspaw

John E. Vincent

Len Lagestee

Mike Fiedler

Nathen Harvey

Patrick Debois

Tom Duffield

Videos

devopsdays on Vimeo

Newsletters

DevOps Weekly

Continue reading →


Chef integration testing with serverspec

Most resources discussing testing with Chef deal with unit testing. Unit testing your Chef recipes is a very good idea, but will only get you so far. You can have 100s of passing unit tests and your Chef converges will still fail. Integration testing can greatly increase the confidence you have in your Chef code. In this post, I’ll walk through how we’re currently doing it at Essess.

First, a clarification on Chef testing terms

I didn’t really understand how chefspec unit tests fit into the Chef testing process until I watched Seth Vargo’s talk at a June meetup on it. If you use Chef and haven’t watched it yet, check it out. He broke it down like this:

Unit testing in Chef is intention testing. When you run your chefspec tests, you are only testing whether you are telling Chef to do what you expected. When is this useful? When you have some complicated...

Continue reading →


Testing Python command line apps

Recently I wrote a Python command line app and had some trouble finding a good example of how to test it using best practices. Some ideas I ran across in my search:

  • Use a 3rd party CLI-testing solution like ScriptTest or a framework that includes testing like pyCLI.

I write my other tests with unittest and run them with nose. I don’t think there’s anything special about a CLI that requires the rest of my team to learn YATT (yet another testing tool).

  • Use sys.argv, subprocess or some combination of both.

This feels very hacky to me. Writing a Python CLI using sys.argv is a little silly considering the more robust options out there. argparse is in the standard library. 3rd party tools like clint and cliff are frameworks that seem promising. A testing solution that plays to the strength of these tools is ideal.

Personally, I use argparse. It does everything I need and its...

Continue reading →


Multi-VM Vagrant the DRY way

So Vagrant is great. If you’re not using it yet, you should check it out. It allows you to develop your applications in the same environment they’ll be running in.

I’ve been using Vagrant and Chef for a little over a year now, but I hadn’t used Ruby much before. I’ve written a lot of Chef cookbooks and Vagrantfiles since then. Writing a multi-VM Vagrantfile is much easier if you lean on Ruby to help you keep them shorter and more readable.

Here’s an example of a Vagrantfile with 3 Ubuntu LTS 12.04 machines:

  • clean - A base box, with nothing installed. For experimentation.
  • mongo - A MongoDB box, provisioned with chef-solo.
  • jenkins - A Jenkins CI box, provisioned with chef-solo.

There are 3 parts to the Vagrantfile.

1: Box definitions

Boxes can be configured with several options, most of them optional. The schema is defined at the top of the file.

...

Continue reading →