Software is of little value if it doesn't work as intended. So to ensure correct operation, software development methodologies have traditionally included a separate phase dedicated solely to testing. In the testing phase, errors and omissions are uncovered and resolved, (hopefully) yielding higher quality code.

While testing after coding can be effective, new, more modern development methodologies incorporate testing directly into development. For example, in eXtreme Programming (XP), writing tests before writing code is a principal practice. According to this test-driven development (TDD) method, and assuming that a software component's tests are comprehensive, a component is unequivocally complete when all tests pass.

Testing code in Python is even more important. Since Python is a completely interpreted language, code is checked for consistency only at runtime. This means that even simple errors -- like a typo in naming a variable -- are only discovered when the code actually runs, either by the developer or the end-user. There are tools (for instance PyChecker and PyLint) that attempt to overcome this problem using static analysis, but because of the completely dynamic nature of the Python language, such analysis can't be fully trusted, since little can be asserted at compile time. (Nonetheless, these tools do have merits, since they can help you catch many common errors easily.)

To test code at runtime, the official Python distribution comes with a module named PyUnit (http://pyunit.sourceforge.net), an implementation of the famous XUnit framework. However, while suitable for testing normal classes and applications, PyUnit lacks some features necessary to test graphical user interfaces (GUI).

So what's a developer to do? Try PyGUIUnitTest, an extension to the PyUnit module that allows you to generate unit tests to exercise your GUI, both aiding your development and catching lurking problems.

GUIs are part of most developers' toolkits nowadays, as most applications usually have windows, buttons, menus, and so on. There are several GUI toolkits available, like Qt, wxWindows, Tkinter, just to name a few, and Python has bindings for most of them.

Although an important part of most applications, the GUI is not typically covered by unit tests. There are many reasons for this, but the most commom are the lack of support in unit testing tools and the the myth that testing GUIs is inherently hard. For instance, a common misconception about GUI testing is that the developer must reproduce user input exactly, clicking the same spot a user would, say, to activate a button. But that's not necessarily the case. For unit testing, the act of activating the button is good enough, and most GUI toolkits provide an easy way to to that without mimicking mouse clicks.

However, while writing GUI tests, the developer must be careful to ensure that the test isn't brittle. That is, the test must be resistant to superficial changes. It shouldn't "break" if a button changes position as a result of a cosmetic change, but should break if pressing the button doesn't do what it used to do anymore. (That's the problem with solutions focused on recording user input as a means of testing the interface: a simple layout change renders many tests useless.)

Let's look at how to develop GUI tests using PyQt and the aforementioned PyGUIUnit.

PyQt is a binding for the Qt GUI toolkit. Qt is extremely portable and provides a native look-and-feel for various plataforms, including Windows, Linux, and Solaris, among others. (More information can be obtained at http://www.trolltech.com, the Trolltech site, the makers of Qt, or at Riverbank Computing, the creators of the Python bindings, at http://www.riverbankcomputing.co.uk/pyqt.

PyGUIUnit (http://www.sourceforge.net/projects/pyguiunit) is a GUI unit testing framework written by Fabio Zadrozny and Bruno da Silva de Oliveira at Engineering Simulation and Scientific Software (ESSS, http://www.esss.com.br). PyGUIUnit is built on top of the PyUnit framework and merges seamlessly with it, allowing you to write tests for GUIs as you would for any other piece of code. PyGUIUnit's current implementation is based on the PyQt framework, but extensions could easily be coded for other GUI toolkits.

Let's begin with a simple example that demonstrates some of the PyGUIUnit framework's capabilities. The example application has one window, which contains a button and a label. When the button is clicked, the text on the label changes to the friendly "Hello World".

To build the GUI, use Qt Designer (Qt's form designer) to create a simple form named HelloWorldDialog that contains a button and label. Then, start coding the test:

import guiunittest as unittest
from HelloWorldDialog import HelloWorldDialog

class TestHelloWorld(unittest.GUITestCase):

(If you're familiar with the unit test module, you'll probably catch that this test case subclasses from GUITestCase instead of the usual TestCase. GUITestCase contains additional logic to handle the subtleties of GUIs and methods to generate user-like input.)

Continuing with the test case...

def setUp(self):
    self.dialog = HelloWorldDialog()
    self.setWidget(self.dialog) # important!

The setUp() method provides the initialization for this test, and the call to setWidget() indicates to the framework what widget is to be tested. (This implies that only one widget can be tested by each GUITestCase subclass.) setWidget() should always be called at setUp() time.

Now, finally write the code to test your dialog:

def testHello(self):
    self.assertEquals('', self.dialog.label.text())
    self.click(self.dialog.button)
    self.assertEquals('Hello World!', self.dialog.label.text())

First, the code checks that the label's contents are empty. Next, it uses the click method supplied by the framework to simulate a user click in the given widget, in this case, the button. Finally, the test verifies that the label's text changed to "Hello World!" as expected.

Adding the traditional "main" idiom to the code produces a test that can be run directly from the command line:

if __name__ == '__main__':
    unittest.main()

Running the test produces:

.
----------------------------------------------------------------------
Ran 1 test in 0.100s
OK

When running a test like this one, Qt's main loop isn't executed as usual. Instead, the event loop is "captured," meaning that events are only processed at certain times, like after calling user input, between tests, or calling processEvents().

The PyGUIUnit framework provides a number of methods to interact with the GUI as a user would. To simulate mouse events, the programmer can use:

def mousePress(self, widget, button=None, position=None, state=None):
    '''Sends a press event for the given widget.

    @param button: the pressed mouse button. Can be LEFT, RIGHT or 
       MIDDLE. If not given, LEFT is assumed.

    @param position: a QPoint or a pair, indicating the position in 
       widget coordinates where the button was pressed. If not given,
       the center of the widget is used.

    @param state: secondary keys. optional, can be SHIFT, 
       CONTROL, ALT.
    '''

def mouseRelease(self, widget, button=None, position=None, state=None):
    '''Sends a mouse release event for the given widget.

    @see: mousePress for the meaning of the arguments.
    '''


def mouseMove(self, widget, position=None, state=None):
    '''Sends a mouse move event for the given widget.

    @see: mousePress for the meaning of the arguments.
    '''


def mouseDrag(self, widget, pressOn, releaseOn, button=None, state=None):
    ''' Makes a drag with the mouse.
    @pressOn: this is the position where the mouse is pressed.
    @releaseOn: this is the position where the mouse is released.
    '''


def click(self, widget, button=None, position=None, state=None):
    '''Acts as if the given widget was clicked. Equivalent to send a
    MousePress followed by a MouseRelease.

    @see: MousePress for the meaning of the arguments.
    '''

def doubleClick(self, widget, button=None, position=None, state=None):
    '''Sends a double-click event to the given widget.

    @see: MousePress for the meaning of the arguments.
    '''

So, to click on a button, you can write:

def testClick(self):
    ...
    self.click(self.mybutton)
    ...

Or, to press the right button on a specific part of a widget, you can use:

def testIt(self):
    ...
    self.mousePress(self.mywidget, position=(32, 10), button=self.RIGHT)

Methods to simulate user text input are also provided:

def keyPress(self, widget, key, state=None):
    '''Sends a key press event to the given widget.

    @param key: a Qt.Key_* constant or a one-char string.

    @param state: secondary keys. optional, can be SHIFT, CONTROL, ALT.
    '''

def keyRelease(self, widget, key, state=None):
    '''Sends a key release event to the given widget.

    @see: keyPress for the meaning of the arguments.
    '''

def type(self, widget, key, state=None):
    '''Acts as if a key was typed in the given widget. Equivalent to a
    KeyPress followed by a KeyRelease.

    @see: keyPress for the meaning of the arguments.
    '''

def typeText(self, widget, text):
    '''Types the text over the given widget.
    '''

Finally, there's a useful method named debugHere(). Meant to be used while developing tests, it gives the control back to Qt's loop, allowing you to interact with the GUI. If a test is failing, add the statement self.debugHere() just before the line that's misbehaving to help debug the test.

As you try to create more GUI unit tests, you'll likely need more power for some simple tasks, such as making sure that some function is called, continuing the tests after execution has been blocked by a modal dialog (remember that the normal execution of the main loop is blocked by the framework), making tests that depend on some generated image (especially if you are using OpenGL), and so on. Let's cover some of those more complex requirements.

from guiunittest import mocks
import os
#other imports omitted


class TestMock(GUITestCase):

    def setUp(self):
        self.win = Dialog()
        # here we replace the function with our mock
        mocks.install(os, system=self.fakeSystem)
        self.setWidget(self.win)


    def tearDown(self):
        # always make sure to uninstall the mocks!
        mocks.uninstall(os)


    def testExecution(self):
        # type the arguments for the external program as a user would
        self.typeText(self.edParams, 'buzz -c -v')
        # clicking in this button should call os.system() with the
        # arguments entered in the edit, but it will actually call our 
        # mock method
        self.click(self.win.button)

    def fakeSystem(self, cmd):
        self.assertEqual(cmd, 'buzz -c -v')

mocks.install(os, system=self.fakeSystem)

class TestTimer(GUITestCase):

    def setUp(self):
        self.win = Dialog()
        self.setWidget(self.win)

    def testTimer(self):

        #first, we set the function to be called
        #after the elapsed time
        self.executeAfterElapsed(self.checkDialog, 300)

        #and only then do we click the button that
        #creates the modal window.
        self.click(self.win.button)

    def checkDialog(self):

        #after the elapsed time this function should be activated.
        #so, first thing we do is checking if the active
        #window is really the one we expected.
        activeWindow = qt.activeWindow()
        self.assert_(isinstance(activeWindow, QDialog))
        self.assert_(activeWindow is not self.win)

        #after everything is checked, we click the ok button
        #to close the dialog and keep going with the test.
        #note that if it is not closed, the test will hang until
        #it is killed.
        self.click(activeWindow.btok)

def assertBMPsEqual(f1, f2, showIfDiff=False, localTolerance=None, 
                    totalTolerance=None, maxDifferentPixels=None,
                    diffOutput=None):
    '''
    This function compares 2 images stored in 2 different files.
    @param f1: first file to compare

    @param f2: second file to compare

    @param showIfDiff: whatever if the images are different, the 
       difference should be shown in an external viewer.

    @param localTolerance: this is a tolerance that can be specified so 
       that the difference within a color is or not considered.
       e.g.: if we have in rgb (0,0,0) and (3,2,4) and a tolerance
           of 3 the first 2 values will be considered equal and the last 
           will not. However, if we had a tolerance of 4, all would be 
           considered equal.

    @param totalTolerance: this is the total tolerance that is accepted 
       within a pixel.
       e.g.: if we have in rgb (0,0,0) and (1,1,1) and a tolerance of 
       3 the pixel will be considered equal, however, if the tolerance 
       specified was 2, it wouldn't.

    @param maxDifferentPixels: This is the number of pixels that can be 
       different in an image; note that only the pixels that have a 
       failure in the tolerance check are considered as different.

    @param diffOutput: used to print debuggin messages. 
       Must be a file-like object.

    @return: True if the images are considered equal and false otherwise.
    '''

def setWidget(self, widget, show=None, wait=None):
    '''Must be called in the setUp() method, giving the test widget.

    @param show: If show() should be called on the GUI. Set to False 
       if you don't want to see the GUI running.

    @param wait: How long to wait between events, in seconds.
    '''

Murphy's Law says, "If there's a possibility that something will go wrong, it will." Its corollary in software development might say, "If there's a possibility that code will break in the future, it will." Unit tests can help prevent surprises. And while writing unit tests for GUIs can be a challenge at first, the effort is more than worthwhile. Think of unit tests as insurance.