Testing with unittest

In this post i introduce the unittest module and set up a few basic tests for the code I have developed so far. While setting up the new testing sure enough i were surprised by an error and had to restructure one test. While the new test works, I do not think the last test case is very good, so suggestions are welcome …

So far I have added a little test function or two in the top of the interp module and changed it for every new feature I wanted to implement in the interpreter.

This has the advantage of being able to quickly try things out while writing interp.py, but it has important disatvantages:

• Tests are in same file as “production” code.
• I lose all the previous tests.
• No standard way to regularly run the test.

In order to improve the testing, I am going to use the unittest module of the Python standard library.

unittest provides the TestCase class which you subclass to add your own test methods. Further there is a TestSuite class which allows you to organize and run your test cases. For the moment I am not going to use TestSuite instances but will let unittest.main take care of providing a simple command line interface for my tests.

if __name__ == '__main__':
    unittest.main()

Will provide the command line interface

~/interp$ python interp_test.py -b
...
----------------------------------------------------------------------
Ran 3 tests in 0.001s

OK

The -b switch suppresses stdout for successful tests.
There is also -v switch which will cause the test runner to name each test.

My test from the first post is implemented like this, and the one for the for loop is very similar. All tests I have so far just test that the returned value of the test function is the same in Python and in my own interpreter.

class TestRetrunValues(unittest.TestCase):
    """
    These tests are verifying that the return values
    of some functions are the same in native Python and
    in my interpreter.
    """
    def test_basic_operations(self):
        """
        Testing the first example
        """
        def test():
            a = 2
            b = a + 4
            return (a + 1) * (b - 2)

        expected = test()
        realized = interp.execute(test.func_code, {})

        self.assertEqual(expected, realized)

When I copied the test for calling a function into the test module I got surprisingly an error. Surprisingly because the same code has worked when it was directly in the interp.py.

    def test_call(self):
        """
        test from third post: calling a function
        """
        def test():
            return square(4) + square(3)

        def square(n):
            return n * n

        expected = test()
        realized = interp.execute(test.func_code, test.func_globals)

        self.assertEqual(expected, realized)

======================================================================
ERROR: test_call (__main__.TestRetrunValues)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "interp_test.py", line 50, in test_call
    realized = interp.execute(test.func_code, test.func_globals)
  File "/home/leonhard/interp/interp.py", line 148, in execute
    raise Exception('Unknown Opcode %d (%s)' % (bc, dis.opname[bc]))
Exception: Unknown Opcode 136 (LOAD_DEREF)

----------------------------------------------------------------------

What happend, and why is it different from when the test was in the interp.py file?

The difference is that now the test and square functions are not defined in the global scope. Therefore the test function, when looking for square, will not search for it in the global namespace but in its closure. I will not go into this topic more right now, what counts is that for this test to succeed I will first need to extend my interpreter more. So for the moment I am going to re-write this test case.

At the same time I do not want to lose this testcase, but I do not want to have my testing fail for this. unittest provides a decorator, @skip, which will disable the test method with a message. There are also more sophisticated, conditional skips available.

    @unittest.skip('closures (e.g. LOAD_DEREF) not yet implemented')
    def test_call(self):

In verbose mode unittest will now display the skip message and the overall test result is “OK (skipped=1)”

For implementing the amended test case I define the executed functions on module level. Actually I am not really satisfied with this solution as the test case is now split onto different parts of the interp_test.py file. If anyone could propose a better way to define this test I would appreciate much your comment!

def _call_global():
    '''
    A test function for the following unittest. 
    Must be defined in module level
    '''
    return _square(4) + _square(3)

def _square(n):
    return n * n

class TestRetrunValues(unittest.TestCase):
    def test_call_global(self):
        """
        test from third post: calling a function, at module level
        """
        test = _call_global
        expected = test()
        realized = interp.execute(test.func_code, test.func_globals)

        self.assertEqual(expected, realized)

Edit: Sometimes the answer is just too obvious: if I want square to be treated as a global, i can simply use the global keyword, of course:

    def test_call(self):
        global square
        def test():
            return square(4) + square(3)

        def square(n):
            return n * n
        [...]

You can have a look at interp_test.py developed in this post on github.