Excerpt from Easier, faster testing
How do we actually choose the conditions that we test?
For a function like
_and, it’s easy. There are only four possible inputs:
- True, True
- True, False
- False, True
- False, False
We’ll test them all. But what about a function
head that returns the first element of a list? We can’t test every possible list, and, intuitively, it would be incredibly wasteful even to try. So how do we choose which lists are important?
First, we use formal methods again. We annotate the types by creating a generic type variable with
The type of elements in the list is generic, so it doesn’t matter what type we use. We could even let
NoneType, which has only one value (
None), and use lists
[None, None], … .
That turns out not to be what we want because we would not be able to distinguish if we are accidentally taking the second or last element of the list instead of the first. For other functions, like finding the length of a collection, testing with
NoneType is very useful.
For this case, though, we want a bigger type that we can distinguish values of, like
int. But which values of
List[int] are important?
This function only touches the first element of a list (if it exists). Passing a list of fifty one elements to this function is going to be basically the same as passing a list of fifty two elements or a hundred and two elements. On the other hand, the empty list, list with one element, and list with two elements are probably all things we want to test.
This is a pattern we will see again and again. If we consider the space of possible inputs to a function, most functions have small boundaries in that space where their behavior changes dramatically, connected by broad stretches where things are pretty much the same. This gives us an heuristic:
Principle 2: Carefully test the boundary. Sample the bulk.
With that in mind, let’s write a test plan for
import unittest from head import head class TestHead(unittest.TestCase): # Boundary: #  -> raises ValueError #  -> 522 # [5, 2] -> 5 # Bulk: # [128, 53, 921, 1022, 41] -> 128 # [1, 5, 3, 12, 8, 1, 3, 2, 1, 5, 3, 19] -> 1 def test_empty_list(self): with self.assertRaises(ValueError): # head behaves quite differently on , # so we can't lump it in with the other # cases without being ridiculously # complicated. head() def test_head(self): cases = [ (, 522), ([5, 2], 5), ([128, 53, 921, 1022, 41], 128), ([1, 5, 3, 12, 8, 1, 3, 2, 1, 5, 3, 19], 1), ] for input, expected in cases: with self.subTest(input=input): found = head(input) self.assertEqual(expected, found)
This is the simplest case. The full course expands the boundary and bulk heuristic to multiple arguments, complex, recursive types and external services, along with introducing other techniques and heuristics.