Smother Documentation

Smother is a tool to measure “who tests what” in a Python test suite. It uses coverage.py to track coverage separately for each test. This can be used to:

  • Lookup which tests visit a particular line of sourcecode.
  • Perform regression test selection – given a diff of local modifications, enumerate which tests in a lengthy test suite might have broken.
  • Explore the amount of coupling between test and application code.
  • Accelerate fancier test techniques like mutation testing.

Quick Tour

Here’s a tour of using smother on smother’s own test suite

# runs test suite, builds a .smother report
> py.test --smother=smother

# which tests ran line 153 of module smother.python?
> smother lookup smother.python:153

smother/tests/test_cli.py::test_lookup[module]
smother/tests/test_cli.py::test_lookup[name]
smother/tests/test_cli.py::test_lookup[range]
smother/tests/test_cli.py::test_lookup[single]
smother/tests/test_cli.py::test_semantic_flatten
...

# which tests visited the function PythonFile.line_count in that module?
> smother lookup smother.python:PythonFile.line_count

smother/tests/test_cli.py::test_lookup[module]
smother/tests/test_interval.py::test_init_module
smother/tests/test_interval.py::test_module_path

# Hmm I've modified something locally
> git diff
@@ -175,6 +175,7 @@ class PythonFile(object):

     @property
     def line_count(self):
+        assert False, "This is a new bug!"
         return len(self.lines)

# What might have broken?
> smother diff

smother/tests/test_cli.py::test_lookup[module]
smother/tests/test_interval.py::test_init_module
smother/tests/test_interval.py::test_module_path

# Run those tests!
smother diff | xargs pytest  # 3 failures

# Dump the report to a csv of <source context, test context>
smother csv report.csv

# Build a vanilla .coverage file
smother to_coverage && coverage html

Contents:

Indices and tables