Testing Python command line apps

Recently I wrote a Python command line app and had some trouble finding a good example of how to test it using best practices. Some ideas I ran across in my search:

I write my other tests with unittest and run them with nose. I don’t think there’s anything special about a CLI that requires the rest of my team to learn YATT (yet another testing tool).

This feels very hacky to me. Writing a Python CLI using sys.argv is a little silly considering the more robust options out there. argparse is in the standard library. 3rd party tools like clint and cliff are frameworks that seem promising. A testing solution that plays to the strength of these tools is ideal.

Personally, I use argparse. It does everything I need and its API is straightforward.

Case study #

I want to write a Python CLI app that will ping an arbitrary set of EC2 instances on AWS that share a set of tags. It also takes some optional params, the AWS region and AMI.

Here’s the app code:


import argparse

def ping(tags, region=None, ami=None):
    # some AWS/boto code here

def create_parser():
    parser = argparse.ArgumentParser(
        description='Ping a number of servers in AWS based on tags'

        'tags', nargs='+',
        help='Tags to search for in AWS'

        '-R', '--region', type=str, required=False,
        help='AWS region to limit search to'

        '-A', '--ami', type=str, required=False,
        help='AWS AMI to limit search to'

    return parser

def main():
    parser = create_parser()
    args = parser.parse_args()
    ping(args.tags, args.region, args.ami)

if __name__ == '__main__':

Pretty simple: one positional argument, tags, followed by 2 optional named parameters, region and ami. I’ve left the actual implementation blank; not needed for this discussion.

So, how should I test this? Above you can see I split the creation of the parser out to create_parser. I did this to make it easier to test. Here are a couple unit tests:


from ping import create_parser, ping
from unittest import TestCase

class CommandLineTestCase(TestCase):
Base TestCase class, sets up a CLI parser
def setUpClass(cls):
    parser = create_parser()
    cls.parser = parser

class PingTestCase(CommandLineTestCase):
    def test_with_empty_args():
        User passes no args, should fail with SystemExit
        with self.assertRaises(SystemExit):

    def test_db_servers_ubuntu_ami_in_australia():
        Find database servers with the Ubuntu AMI in Australia region
        args = self.parser.parse_args(['database', '-R', 'australia', '-A', 'idbs81839'])
        result = ping(args.tags, args.region, args.ami)
        # Do some othe assertions on the result

In the above test file I set up a CommandLineTestCase class that will add the parser from ping.py before any tests are run. The tests then use the parser object on the class to test any number of arguments passed in. This is where the create_parser function in ping.py is key: I am able to create the parser used in the app and unit tests the same way. I can be sure if my tests pass my app will be have the same way.

I hope this is useful to some of you out there. I think unit tests are sometimes seen as a solved problem and there isn’t much discussion around them. They’re hard to teach because they’re so application specific. I’ll be writing about different approaches to testing in the future.


Now read this

From Zero to Hero, or There and Back Again

You just landed your first software job. You’re excited, this is a great opportunity! You’ll be able to apply that expensive Computer Science degree to something useful and start paying back those students loans. (If you’re self-taught,... Continue →