Testing Python command line apps

Recently I wrote a Python command line app and had some trouble finding a good example of how to test it using best practices. Some ideas I ran across in my search:

I write my other tests with unittest and run them with nose. I don’t think there’s anything special about a CLI that requires the rest of my team to learn YATT (yet another testing tool).

This feels very hacky to me. Writing a Python CLI using sys.argv is a little silly considering the more robust options out there. argparse is in the standard library. 3rd party tools like clint and cliff are frameworks that seem promising. A testing solution that plays to the strength of these tools is ideal.

Personally, I use argparse. It does everything I need and its API is straightforward.

 Case study

I want to write a Python CLI app that will ping an arbitrary set of EC2 instances on AWS that share a set of tags. It also takes some optional params, the AWS region and AMI.

Here’s the app code:

ping.py

import argparse

def ping(tags, region=None, ami=None):
    # some AWS/boto code here
    pass

def create_parser():
    parser = argparse.ArgumentParser(
        description='Ping a number of servers in AWS based on tags'
    )

    parser.add_argument(
        'tags', nargs='+',
        help='Tags to search for in AWS'
    )

    parser.add_argument(
        '-R', '--region', type=str, required=False,
        help='AWS region to limit search to'
    )

    parser.add_argument(
        '-A', '--ami', type=str, required=False,
        help='AWS AMI to limit search to'
    )

    return parser

def main():
    parser = create_parser()
    args = parser.parse_args()
    ping(args.tags, args.region, args.ami)

if __name__ == '__main__':
    main()

Pretty simple: one positional argument, tags, followed by 2 optional named parameters, region and ami. I’ve left the actual implementation blank; not needed for this discussion.

So, how should I test this? Above you can see I split the creation of the parser out to create_parser. I did this to make it easier to test. Here are a couple unit tests:

tests/test_ping.py

from ping import create_parser, ping
from unittest import TestCase

class CommandLineTestCase(TestCase):
"""
Base TestCase class, sets up a CLI parser
"""
@classmethod
def setUpClass(cls):
    parser = create_parser()
    cls.parser = parser

class PingTestCase(CommandLineTestCase):
    def test_with_empty_args():
        """
        User passes no args, should fail with SystemExit
        """                                    
        with self.assertRaises(SystemExit):
            self.parser.parse_args([])

    def test_db_servers_ubuntu_ami_in_australia():
        """
        Find database servers with the Ubuntu AMI in Australia region
        """
        args = self.parser.parse_args(['database', '-R', 'australia', '-A', 'idbs81839'])
        result = ping(args.tags, args.region, args.ami)
        self.assertIsNotNone(result)
        # Do some othe assertions on the result

In the above test file I set up a CommandLineTestCase class that will add the parser from ping.py before any tests are run. The tests then use the parser object on the class to test any number of arguments passed in. This is where the create_parser function in ping.py is key: I am able to create the parser used in the app and unit tests the same way. I can be sure if my tests pass my app will be have the same way.

I hope this is useful to some of you out there. I think unit tests are sometimes seen as a solved problem and there isn’t much discussion around them. They’re hard to teach because they’re so application specific. I’ll be writing about different approaches to testing in the future.

 
360
Kudos
 
360
Kudos

Now read this

How To: Render AWS CloudFormation templates with Docker

If your infrastructure runs on AWS and you’re not yet using CloudFormation, you should give it a go. CloudFormation (from here on, “CFN”) is a powerful member of the AWS toolbox that allows you to declare every part of your... Continue →