Grabbing AWS credentials with bash

Let’s say you’re running on AWS and using IAM to assign your instances AWS keys dynamically. First off, if you’re not doing this you should. Why?

Okay, so we’re all on board: If we’re using AWS we should use IAM roles to provide rotating access keys to our instances.

But what happens when you need those keys? I ran across this problem running unit tests on Jenkins for a Chef cookbook that uses citadel to pull config secrets out of S3. I couldn’t really mock out the citadel call, so I wanted to pass my AWS credentials in via environment variables to chefspec.

We’ll be using the EC2 instance metadata API. Here’s how you do it.

First, we need to grab the name of the IAM role attached to the instance.


You could hard-code this in, but if you’re using CloudFormation to set up IAM, this will change when you rebuild your stacks.

Alright, we’ve got the IAM role name, which we’ll now use to grab the access and secret access keys.

aws_access_key_id=`curl${instance_profile} | grep AccessKeyId | cut -d':' -f2 | sed 's/[^0-9A-Z]*//g'`

aws_secret_access_key=`curl${instance_profile} | grep SecretAccessKey | cut -d':' -f2 | sed 's/[^0-9A-Za-z/+=]*//g'`

Now we have the keys. We can then export then to the environment and run our tests.

export AWS_ACCESS_KEY_ID=${aws_access_key_id}
export AWS_SECRET_ACCESS_KEY=${aws_secret_access_key}

rspec --format RspecJunitFormatter --out chefspec.xml

Great, now our tests pass and we didn’t have to hard-code any credentials into the Jenkins job!

If you’re using boto for Python or fog for Ruby, they will internally query the instance metadata store for these credentials. This is great, and something you can use to your advantage. But sometimes you can’t/don’t want to depend on another library for this. bash and curl to the rescue!


Now read this

Boston DevOps Show and Tell

The March 24 Boston DevOps meetup was a show and tell, a series of short talks from the community on tools and techniques they find helpful in their daily work. Jenkins (2.0) Pipeline as Code - Thomas McGonagle # Tom shares the new... Continue →