NB: Unit Testing with Unittest

Programming for Data Science

Unit Testing

Unit testing is bottom-up approach to testing code.

It assumes that code may be decomposed into elementary units that are independent.

Quality assurance is achieved by testing each of these units for their expected behavior under a variety of conditions.

Role of Functions

The elementary unit of code is usually the function.

Recall that functions should be written to perform relatively simple things — simple, well-defined tasks.

Unit tests are meant to eliminate design-time errors, i.e. those produced by the coder.

Unit tests may also be designed to test whether or not functions perform correctly during run-time, across a range of error-producing conditions.

These conditions include edge cases, invalid user inputs, broken connections, etc.

Integration Testing

Unit testing typically focuses on individual functions in isolation.

To test how groups of related functions interact and perform, we employ integration testing.

Integration testing is more complex, given the combinatorial complexity of coding.

Ultimately, the best way to test interacting functions — and the software system as a whole — is through user testing.

But unit testing establishes a sold foundation on which to conduct these higher-level tests.

Role of Design

It is worth noting that unit testing benefits from a kind of design.

If you design functions that are not well-defined and focused, then you are unlikely to test them effectively.

So part of unit testing is function design in the first place.

A broad area of programming called functional programming implements this kind of design.

So, unit testing goes hand-in-hand with software design.

Benefits of Unit Testing

Developers can work in a predictable way as they develop code.

Developers can write their own unit tests and check themselves as they build.

Unit testing provides rapid responses when testing small changes.

It reduces defects in the newly developed features or reduces bugs when changing the existing functionality.

And it reduces cost of testing, since defects are captured in very early phase.

Finally, it improves design and allows better refactoring of code.

Unit Testing Frameworks

There are many of these available to Python users, including:

Unittest: A full-featured unit testing system, inspired by Java’s JUnit. Part of the Python Standard Library (as an external project it was called PyUnit).
DocTest: A lightweight test framework that allows you to embed tests directly in the docstrings, making it easy to combine code examples and test cases in one place. Part of the Python Standard Library.
PyTest: An alternative to PyUnit and with a simpler syntax.
nose2: A discovery-based framwork that extends unittest (successor to nose).
Testify: A Pythonic testing framework compatible with unittest tests.
Robot: A framework for test automation and robotic process automation (not just for Python).
Behave: A framework for Behavior-Driven Development.
Hypothesis: A powerful property-based testing.

— Adapted from https://wiki.python.org/moin/UnitTests

We will focus on Unittest.

The Basic Idea

The Unittest framework provides you with a testing class that you can inherit.

This class contains a bunch of assert methods.

The basic idea is to write functions that test other functions by using these assert methods.

These functions are put in another file that calls your module.

This prevents you from peppering your code with assert statements.

It aslo allows a clean separation of concerns.

Assert Methods

Unittest provides many assert methods — see this cheat sheet for more.

These are wrappers around Python’s built-in assert function.

Assert methods compare actual with expected results.

Some commonly used assert methods are:

  • assertTrue(): Used to test if your function produces an actual result that matches an expected result.
  • assertFalse(): Used to test if the actual result fails in an expected way.
  • assertEqual(): Used to test if actual and expected results are equal.
  • assertRaises(): Used to test if raise statement is raising the correct exception.

The Basic Pattern

The Unittest framework works as follows:

  1. Choose a package or class that you want to test.

  2. Create a .py file to put your unittest code.

  3. In the file, create a class that is a subclass of unittest.TestCase.

  4. In that class write methods that are designed to test the behavior of methods in the code you want to test.

  5. Run the script from the command line and see the results.

  6. Update the script as you create new methods or refactor existing ones.

In the test class:

  • Each test method focuses on one behavior of one method (or function).

  • There can be many test methods for each target method.

  • Each test method name must be prefixed by test_.

  • Tests are executed in alphabetical order, so name them in the order you want them executed.

  • Each test makes use of an assert method.

  • You want all tests to pass, so if you want to test if something breaks, you return True for a False condition.

Let’s look at some examples.

Example 1

Consider the case of this simple static class, stored in demo/math_ops.py, to perform basic arithmetic operations.

class MathOps:

    def add(a, b):
        return a + b
        
    def subtract(a, b):
        return a - b

    def multiply(a, b):
        return a * b

    def divide(a, b):
        if b == 0:
            raise ValueError("Cannot divide by zero")
        return a / b
    
if __name__ == '__main__':
    print("This is the MathOps class. It's meant to be imported into another module.")
    print("But since you called it directly, I'll show you how it works!")
    x = 5
    y = 10
    print("5 + 10 =", MathOps.add(x, y))
    print("5 - 10 =", MathOps.subtract(x, y))
    print("5 * 10 =", MathOps.multiply(x, y))
    print("5 / 10 =", MathOps.divide(x, y))
!python demo/math_ops.py
This is the MathOps class. It's meant to be imported into another module.
But since you called it directly, I'll show you how it works!
5 + 10 = 15
5 - 10 = -5
5 * 10 = 50
5 / 10 = 0.5

Here is an example unittest file, stored in demo/math_ops_test.py, to test the methods in the class:

import unittest
from math_ops import MathOps 

class TestMathOperations(unittest.TestCase):
    
    def test_add(self):
        self.assertEqual(MathOps.add(2, 3), 5)
        self.assertEqual(MathOps.add(-1, 1), 0)
        self.assertEqual(MathOps.add(-1, -1), -2)

    def test_subtract(self):
        self.assertEqual(MathOps.subtract(5, 3), 2)
        self.assertEqual(MathOps.subtract(1, 1), 0)
        self.assertEqual(MathOps.subtract(-1, -1), 0)

    def test_multiply(self):
        self.assertEqual(MathOps.multiply(2, 3), 6)
        self.assertEqual(MathOps.multiply(-2, 3), -6)
        self.assertEqual(MathOps.multiply(0, 5), 0)

    def test_divide(self):
        self.assertEqual(MathOps.divide(6, 3), 2)
        self.assertEqual(MathOps.divide(7, 2), 3.5)
        self.assertRaises(ValueError, MathOps.divide, 1, 0)

if __name__ == '__main__':
    unittest.main()

To run the test file, we put the two files in the same directory and run the test file from the command line.

!python demo/math_ops_test.py
....
----------------------------------------------------------------------
Ran 4 tests in 0.000s

OK

To save the results of our test to a file, we can do this:

!python demo/math_ops_test.py 2> demo/math_ops_results.txt

We use 2> because the messages that unittest prints are error messages on Unix.

We can see our results using more:

!more demo/math_ops_results.txt
....
----------------------------------------------------------------------
Ran 4 tests in 0.000s

OK

Example 2

In this example, we create a class called Student and save it in a local file called demo/student.py.

class Student:
    
    # constructor
    def __init__(self, name, courses=None):
        self.name = name 
        self.courses = [] if courses is None else courses 
        self.num_courses = len(self.courses)
        
    # enroll in a course
    def enroll_in_course(self, course_name): 
        self.courses.append(course_name)
        self.num_courses += 1

Then we create a companion test file for our class, saving it in a file called demo/student_test.py.

from student import Student
import unittest

class EnrollInTestCase(unittest.TestCase): 
    
    def test_is_incremented_correctly(self):
        """
        Test if enrollInCourse() method successfully increments the
        num_courses attribute of the Student object 
        """

        student1 = Student('Katherine', ['DS 5100'])
        student1.enroll_in_course("CS 5050")
        student1.enroll_in_course("CS 5777")
        print('Course names:', student1.courses)
        print('Course count:', student1.num_courses)
        
        expected = 3
        self.assertEqual(student1.num_courses, expected)
    
if __name__ == '__main__':
    unittest.main()

To run our tests, we run the test file from the command line:

!python demo/student_test.py
Course names: ['DS 5100', 'CS 5050', 'CS 5777']
Course count: 3
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

Again, to capture the results, we do this:

!python demo/student_test.py 2> demo/student_results.txt
Course names: ['DS 5100', 'CS 5050', 'CS 5777']
Course count: 3

Note that program still outputs our print statements.

Here we capture the print calls in the module and only shows the errors.

!python demo/student_test.py > demo/student_results1.txt
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK