The first function that you’ll test is above_freezing from :
| | def above_freezing(celsius: float) -> bool: |
| | """Return True iff temperature celsius degrees is above freezing. |
| | |
| | >>> above_freezing(5.2) |
| | True |
| | >>> above_freezing(-2) |
| | False |
| | """ |
| | |
| | return celsius > 0 |
In that section, you ran the example calls from the docstring using the doctest module. But you’re missing a test—what happens if the temperature is zero? In the next section, you’ll write another version of this function that behaves differently at zero, and we’ll discuss how your current set of tests is incomplete.
Before writing your testing code, you must decide which test cases to use. Function above_freezing takes one argument, a number, so for each test case, you need to choose the value of that argument. There are billions of numbers to choose from, and you can’t possibly test them all, so how do you decide which values to use? For above_freezing, there are two categories of numbers: values below freezing and values above freezing. Pick one value from each category to use in your test cases.
Looking at it another way, this particular function returns a Boolean, so you need at least two tests: one that causes the function to return True and another that causes it to return False. That’s what you already did in the example function calls in the docstring.
In the docstring of above_freezing, the first example call uses 5.2 as the value of the argument, and since that value is above freezing, the function should return True. This test case represents temperatures above freezing. Choose that value from among the billions of possible positive floating-point values; any one of them would work just as well. For example, you could have used 100.6, 29, 357.32, or any other number greater than 0 to represent the “above freezing” category.
The second example call uses -2, which represents temperatures below freezing. As before, you could have used -16, -294.3, -56.97, or any other value less than 0 to represent the “below freezing” category, but let’s chose to use -2. Again, your choice is arbitrary.
Are you missing any test case categories? Imagine that you had written your code using the >= operator instead of the > operator:
| | def above_freezing_v2(celsius: float) -> bool: |
| | """Return True iff temperature celsius degrees is above freezing. |
| | |
| | >>> above_freezing_v2(5.2) |
| | True |
| | >>> above_freezing_v2(-2) |
| | False |
| | """ |
| | |
| | return celsius >= 0 |
Both versions of the function produce the expected results for the two docstring examples; however, the code differs from before, and it won’t produce the same result in all cases. You neglected to test one category of inputs: temperatures at the freezing mark. Test cases like the one at the freezing mark are often referred to as boundary cases, as they lie on the boundary between two distinct possible behaviors of the function (in this case, between temperatures above freezing and those below freezing). Experience shows that boundary cases are much more likely to contain bugs than other cases, so it’s always worth identifying them and testing them.
Sometimes there are multiple boundary cases. For example, if you had a function that determined the state of water—solid, liquid, or gas—then the two boundary cases would be the freezing point and the boiling point.
To summarize, the following table displays each category of inputs, the value you chose to represent that category, and the value you expect the function to return in that case:
| Test Case Description | Argument Value | Expected Return Value |
|---|---|---|
| Temperatures above freezing | 5.2 | True |
| Temperatures below freezing | -2 | False |
| Temperatures at freezing | 0 | False |
Now that all categories of inputs are covered, you need to run the third test. Running the third test in the Python shell reveals that the value returned by above_freezing_v2 isn’t False, which is what you expected:
| | >>> above_freezing(0) |
| | False |
| | >>> above_freezing_v2(0) |
| | True |
It took three test cases to cover all the categories of inputs for this function, but three isn’t a magic number. The three tests had to be carefully chosen. If the three tests had all fallen into the same category (say, temperatures above freezing: 5, 70, and 302), they wouldn’t have been sufficient. It’s the quality of the tests that matters, not the quantity.
Once you decide which test cases are needed, you can use one of two approaches that you’ve learned about so far to test the code. The first one is to call the functions and read the results yourself to see if they match your expectations. The second is to run the functions from the docstring using the module doctest. The latter approach is preferable because the comparison of the actual value returned by the function to the expected value is done by the program, not by a human, making it faster and less prone to errors.
In this section, you’ll study another of Python’s modules, unittest. A unit test exercises just one isolated component of a program. As you did with doctest, you’ll use the unittest module to test each function in your module independently of the others. This approach contrasts with system testing, which examines the system’s behavior as a whole, just as its eventual users will.
Test your functions by writing subclasses of the unittest’s TestCase class
In , you learned how to write classes that inherit code from others. Now, you’ll write test classes that inherit from the unittest.TestCase class. Your first test class tests the function above_freezing:
| | import unittest |
| | import temperature |
| | |
| | |
| | class TestAboveFreezing(unittest.TestCase): |
| | """Tests for temperature.above_freezing.""" |
| | |
| | def test_above_freezing_above(self): |
| | """Test a temperature that is above freezing.""" |
| | |
| | expected = True |
| | actual = temperature.above_freezing(5.2) |
| | self.assertEqual(expected, actual, |
| | "The temperature is above freezing.") |
| | |
| | def test_above_freezing_below(self): |
| | """Test a temperature that is below freezing.""" |
| | |
| | expected = False |
| | actual = temperature.above_freezing(-2) |
| | self.assertEqual(expected, actual, |
| | "The temperature is below freezing.") |
| | |
| | def test_above_freezing_at_zero(self): |
| | """Test a temperature that is at freezing.""" |
| | |
| | expected = False |
| | actual = temperature.above_freezing(0) |
| | self.assertEqual(expected, actual, |
| | "The temperature is at the freezing mark.") |
| | |
| | unittest.main() |
The name of your new class is TestAboveFreezing, and it’s saved in the file test_above_freezing.py. The class has three methods, one for each test case. Each test case follows this pattern:
| | expected = the value you expect will be returned |
| | actual = call on the function being tested |
| | self.assertEqual(expected, actual, "Error message in case of failure") |
In each test method, there is a call to the assertEqual method, which has been inherited from the unittest.TestCase class. To assert something is to claim that it is true; here, you are asserting that the expected value and the actual value should be equal. Method assertEqual compares its first two arguments (which are the expected return value and the actual return value from calling the function being tested) to see whether they are equal. If they aren’t equal, the third argument, a string, is displayed as part of the failure message.
At the bottom of the file, the call to the unittest.main method executes every method that begins with the name test.
When the program in test_above_freezing.py is executed, the following results are produced:
| | ... |
| | ---------------------------------------------------------------------- |
| | Ran 3 tests in 0.000s |
| | |
| | OK |
The first line of output has three dots, one dot per test method. A dot indicates that a test was run successfully—that the test case passed.
The summary after the dashed line indicates that unittest found and ran three tests, completed the task in under a millisecond, and all results as successful (OK).
If your faulty function above_freezing_v2 were renamed above_freezing and your test_above_freezing unit test program was rerun, instead of three passes (as indicated by the three dots), there would be two passes and a failure:
| | .F. |
| | ====================================================================== |
| | FAIL: test_above_freezing_at_zero (__main__.TestAboveFreezing) |
| | Test a temperature that is at freezing. |
| | ---------------------------------------------------------------------- |
| | Traceback (most recent call last): |
| | File "test_above_freezing.py", line 30, in test_above_freezing_at_zero |
| | "The temperature is at the freezing mark.") |
| | AssertionError: False != True : The temperature is at the freezing mark. |
| | |
| | ---------------------------------------------------------------------- |
| | Ran 3 tests in 0.001s |
| | |
| | FAILED (failures=1) |
The F indicates that a test case failed. The error message indicates that the failure occurred in the method test_above_freezing_at_zero. The type of the error is AssertionError, which suggests that when you asserted the expected and actual values should be equal, you were incorrect.
The expression False != True comes from your call on assertEqual: variable expected was False, variable actual was True, and of course, those aren’t equal. Additionally, the string that was passed as the third argument to assertEqual is part of that error message: "The temperature is at the freezing mark."
Notice that the three calls on assertEqual were placed in three separate methods. You could have put them all in the same method, but that method would have been considered a single test case. That is, when the module was run, you would see only one result; if any of the three calls on assertEqual failed, the entire test case would have failed. Only when all three passed would you see the coveted dot.
As a rule, each test case you design should be implemented in its own test method.
Now that you’ve seen both doctest and unittest, which should you use? We prefer unittest, for several reasons:
For large test suites, it is nice to have the testing code in a separate file rather than in a very long docstring.
Each test case can be in a separate method, so the tests are independent of each other. With doctest, the changes to objects made by one test persist for subsequent tests, so more care needs to be taken to properly set up the objects for each doctest test case to ensure they are independent.
Because each test case is in a separate method, you can write a docstring that describes the test case tested so that other programmers understand how the test cases differ from each other.
The third argument to assertEqual is a string that appears as part of the error message produced by a failed test, which helps provide a better description of the test case. With doctest, there is no straightforward way to customize the error messages.