Note: I’ve found a better way to visually verify that all tests are running. Check out this post on Jenkins to see how I’m now working with my tests. The below post is still useful as a reference on how to set up UnitTest++ in Qt Creator, also when using Jenkins.*

Note 2: A new and, in my opinion, better project structure is shown in this new post.*

Note 3: See this post for the same project structure using the even better Catch testing framework.*

When you want to make sure that your code is working properly, it is a good idea to divide it into smaller, independent pieces that may be tested individually. A unit test is a short  code that tests a smallest possible portion of your application. It is a good idea to write tests as you go and it can even be useful to write a test before you even implement the function that will be tested.

With the combination of UnitTest++ and Qt Creator, I’m now able to write unit tests while working on the code and also have some nice visual indication about failed tests during the build step of my project:

qt-creator-unit-tests1

During my search for a good setup for testing my applications I realized there were a few needs that I wanted to satisfy:

  1. Creating new unit tests should be dead-easy to do. I want to spend as little time as possible on reading documentation about the testing framework.
  2. Tests should be run immediately after a new build of the source code, automatically and without the extra hassle to remember to run the tests.
  3. Tests should provide visual feedback with an easy way to get to where the tests fail. This should show up in the IDE or some other useful GUI tool.
  4. The testing framework should be fairly easy to install, especially on Ubuntu. This is because I want to be able to promote it to my fellow students.

For a long time I have been using the QtTestlib. It is fairly quick to use, but I have failed to make the tests give me the visual feedback I wanted. In addition it requires the Qt library to run, which makes it hard to advocate to my fellow students that are not using Qt already.

UnitTest++

The UnitTest++ framework is extremely simple to use. To download it in Ubuntu, all you have to do is

sudo apt-get install libunittest++-dev

and create a new C++ file the following way:

#include <unittest++/UnitTest++.h>

TEST(WillFail) {
    CHECK(false);
}

int main()
{
    return UnitTest::RunAllTests();
}

When you compile, all you need to do is to link to the library. Add the following to your .pro file:

LIBS += -lunittest++

You may write as many tests as you like and check that they are successful with the CHECK macro.

Read more about the UnitTest++ library.

The complete solution: UnitTest++ and Qt Creator

To get the most out UnitTest++ it is a good idea to integrate its output into the Qt Creator IDE. The way I have set this up in Qt Creator is with subprojects. One for the main project, which again is split into the app itself and a library, and one for the tests. In addition, I have a helper project file, named defaults.pri. The structure of the project is like this:

MyProject
├─ MyProject.pro
├─ defaults.pri
├─ .qmake.conf
├─ src/
│  ├─ src.pro
│  ├─ app/
│  │  ├─ app.pro
│  │  └─ main.cpp
│  └─ libs
│     ├─ libs.pro
│     └─ myclass.cpp
└─ tests/
   ├─ tests.pro
   └─ main.cpp

The .qmake.conf file is a helper file that defines a few things that we will use in the subprojects. This is only available in Qt5, and looks something like this:

TOP_PWD=$$PWD
TOP_OUT_PWD=$$shadowed($$PWD)

The main project file, MyProject.pro will now be based on a subdirs template, and may look like this:

TEMPLATE=subdirs
SUBDIRS=src tests
CONFIG+=ordered

The CONFIG+=ordered statement makes sure that the src project is compiled before the tests.

The src.pro file looks similar:

TEMPLATE=subdirs
SUBDIRS=libs app
CONFIG+=ordered

The src directory contains the actual project, and is split into app and libs. In app, I now only have a main.cpp file, because the app is basically just something that uses everything in the libs folder. It will depend on the shared compiled library from libs, and app.pro would look something like this:

include(../../defaults.pri)
TEMPLATE = app
SOURCES = main.cpp

In the libs folder, I have myclass.cpp, that contains the MyClass class, and is what I want to test. The libs.pro needs to compile to a library, so that it may be used both by app and tests, and could look something like this:

include(../../defaults.pri)
TEMPLATE = lib
TARGET = myapp
SOURCES = myclass.cpp
HEADERS = myclass.h

What this class does is not so interesting, it could be anything.

In the tests folder I have simply added a main.cpp file which could contain the following:

#include <unittest++/UnitTest++.h>
#include <src/myclass.h>

TEST(MyMath) {
    MyClass my;
    CHECK(my.addition(3,4) == 7);
}

int main()
{
    return UnitTest::RunAllTests();
}

This test will fail because my implementation of MyClass::addition is completely wrong:

class MyClass {
public:
    double addition(double a, double b) {
        return a * b;
    }
};

Note that I’m including MyClass by through <src/myclass.h>. To be allowed to do this, I have included a defaults.pri file that helps me resolve the correct directory of the source files. The defaults.pri file contains this:

# Directories
INCLUDEPATH += $$TOP_PWD/src/libs
SRC_DIR = $$TOP_PWD

And in the tests.pro file I have:

include(defaults.pri)
TEMPLATE = app

CONFIG   += console
CONFIG   -= app_bundle
CONFIG   -= qt

SOURCES += main.cpp

LIBS += -lunittest++ -L$$TOP_OUT_PWD/src/libs -lmyapp

The final two lines will search for any .cpp file in your source directory and add them to your test project. This may be a problem if you have source files hanging around that are not in use by your Qt project, so you may just replace these lines with all your sources explicitly.

If the main program uses some libraries that the test project now will also need, it is very useful to have this common defaults.pri file which may be included in both tests.pro and src.pro.

Getting that visual output

With the above, I’m now able to run the tests executable to get some nice terminal output:

../../MyProject/tests/main.cpp:72: warning: Failure in MyMath: my.addition(3,4) == 7
FAILURE: 1 out of 1 tests failed (1 failures).
Test time: 0.00 seconds.

However, I would like this to show up without having to explicitly run the tests executable. To do this, I have added a Custom Build Step in Qt Creator after the Make step in MyProject.

Do this by selecting Projects > Build Steps > Add Build Step with the following options:

Command: ./tests/tests
Arguments: 1>&2
Working directory: %{buildDir}

If you don’t want the tests to completely fail your build, you may replace the errors with warnings instead by using the following options instead:

Command: ./tests/tests
Arguments:  | sed s/error:/warning:/ 1>&2
Working directory: %{buildDir}

It should look something like this:

qt-creator-build-step}

This will invoke the command line tool sed after running the tests executable to replace all strings with “error:” to “warning:”. Qt Creator will then just list the failures in the Issues tab and start your application even if the tests fail.

The best part

With this up and running, you should get some output like this whenever a test fails:

qt-creator-unit-tests1

And the best thing about it is that you can even click the line to go directly to the test that fails. Isn’t that just great?

Other frameworks

I also came across Boost.Test and GoogleTest. The test framework from Boost seemed to be a bit too much hassle to install and the documentation was a bit outdated. It is also said that the code would build slower due to it being a header-only framework.

GoogleTest was likely the best alternative to UnitTest++, but installing it on Ubuntu was a bit more hassle, which made the choice easy. On the other hand, GoogleTest seems to be a more advanced framework, so I may switch sometime in the future.