Testing Python command line apps
Recently I wrote a Python command line app and had some trouble finding a good example of how to test it using best practices. Some ideas I ran across in my search:
- Use a 3rd party CLI-testing solution like ScriptTest or a framework that includes testing like pyCLI.
I write my other tests with
unittest and run them with nose. I don’t think there’s anything special about a CLI that requires the rest of my team to learn YATT (yet another testing tool).
subprocessor some combination of both.
This feels very hacky to me. Writing a Python CLI using sys.argv is a little silly considering the more robust options out there.
argparse is in the standard library. 3rd party tools like clint and cliff are frameworks that seem promising. A testing solution that plays to the strength of these tools is ideal.
Personally, I use
argparse. It does everything I need and its API is straightforward.
Case study #
I want to write a Python CLI app that will ping an arbitrary set of EC2 instances on AWS that share a set of tags. It also takes some optional params, the AWS region and AMI.
Here’s the app code:
import argparse def ping(tags, region=None, ami=None): # some AWS/boto code here pass def create_parser(): parser = argparse.ArgumentParser( description='Ping a number of servers in AWS based on tags' ) parser.add_argument( 'tags', nargs='+', help='Tags to search for in AWS' ) parser.add_argument( '-R', '--region', type=str, required=False, help='AWS region to limit search to' ) parser.add_argument( '-A', '--ami', type=str, required=False, help='AWS AMI to limit search to' ) return parser def main(): parser = create_parser() args = parser.parse_args() ping(args.tags, args.region, args.ami) if __name__ == '__main__': main()
Pretty simple: one positional argument, tags, followed by 2 optional named parameters, region and ami. I’ve left the actual implementation blank; not needed for this discussion.
So, how should I test this? Above you can see I split the creation of the parser out to
create_parser. I did this to make it easier to test. Here are a couple unit tests:
from ping import create_parser, ping from unittest import TestCase class CommandLineTestCase(TestCase): """ Base TestCase class, sets up a CLI parser """ @classmethod def setUpClass(cls): parser = create_parser() cls.parser = parser class PingTestCase(CommandLineTestCase): def test_with_empty_args(): """ User passes no args, should fail with SystemExit """ with self.assertRaises(SystemExit): self.parser.parse_args() def test_db_servers_ubuntu_ami_in_australia(): """ Find database servers with the Ubuntu AMI in Australia region """ args = self.parser.parse_args(['database', '-R', 'australia', '-A', 'idbs81839']) result = ping(args.tags, args.region, args.ami) self.assertIsNotNone(result) # Do some othe assertions on the result
In the above test file I set up a CommandLineTestCase class that will add the parser from
ping.py before any tests are run. The tests then use the parser object on the class to test any number of arguments passed in. This is where the
create_parser function in
ping.py is key: I am able to create the parser used in the app and unit tests the same way. I can be sure if my tests pass my app will be have the same way.
I hope this is useful to some of you out there. I think unit tests are sometimes seen as a solved problem and there isn’t much discussion around them. They’re hard to teach because they’re so application specific. I’ll be writing about different approaches to testing in the future.