## Experimental Qt and QML in the browser

Update: You might also be interested in the discussion on Hacker News.

I was amazed when I read Morten Johan Sørvig’s blog post about bringing Qt to Native Client (NaCl) and Emscripten in September last year. I have been following multiple efforts to bring Qt to the web for some time. Mostly because I wanted to make my own Qt projects easily available to others, but also because I find the whole concept of running C++ code in a browser fascinating.

This is definitely not the first time someone has worked on bringing Qt to the browser. Qt4 was ported to Emscripten by Simon St James and published with a lot of examples. However, it seems the port was discontinued when Qt5 came along due to the changes that came with that transition. In the meantime, Lauri Paimen, Anton Kreuzkamp and many others have contributed to qmlweb, a QML interpreter written from scratch in JavaScript. That project does however not make it easy to use any C++ classes you may have defined. And most of my projects are QML/C++ hybrids.

These projects brought something important to the table and took us one step closer to writing QML for the browser. Personally I find QML very enjoyable because of its clean and intuitive structure in comparison to other options like XML. And the ability to easily bind GUI elements to high-performance C++ code has made Qt my favorite framework for many projects.

I tried to port Qt5 to Emscripten myself about a year ago. That turned out to be a real challenge. I based my work on the changes Simon St James made to Qt4 and tried to figure out how to make the same changes in Qt5. After a couple of weeks, I got to the point where I had QtCore running and QtDeclarative (QML) compiling, but other things caught up with me and I had to leave that project behind. The biggest disappointment was that I never got around to making any demos.

I decided to pick up the ball a couple of months ago This time I knew about Morten Johan Sørvig’s efforts and asked him to give me some info on the status of his port. At that point he had support for compiling to NaCl ready (that means Chrome only) and he had already started porting to Emscripten using pepper.js as glue between Emscripten and NaCl. There was still a bit left to do before any examples could be up and running in Firefox. I decided to give it a go and see if I could contribute with something.

A couple of weeks later, I managed to get Qt running in Firefox with a number of QML examples. The process was a lot of fun as I learned much about what happens inside Qt’s sources. It was interesting to see how often I could write many lines of code and rebuild multiple times only to arrive at a one-line change that fixed the issue at hand. I was really happy when I finally could send a patch to Qt’s NaCl branch. The main changes were to Qt’s event loop, which would hang if control wasn’t returned to the browser. I had to add a couple of hacks to keep Qt’s mutexes from locking up everything (this needs to be properly fixed later, perhaps by creating pthread stubs). And OpenGL needed to be explicitly set to OpenGL ES2 in addition to a bunch of smaller modifications.

Below are two resulting demos of QML running in the browser. I’m amazed with the performance both in Firefox and Chrome. Just be aware that the size of the examples is quite big because no optimizations have been done to reduce the size of the binary JavaScript. Further, there may be lots of bugs and some browsers where the examples don’t work. Remember that this is still very experimental and only a small taste of what may be possible with Qt in the browser in the future:

What amazes me the most is the fact that we are now running Qt’s own JavaScript engine, which is written in C++ and compiled to JavaScript with Emscripten, inside the browser’s own JavaScript engine! I really want to see if we can get QML’s WebEngineView running inside itself inside Firefox. That’s a step on the way to true inception, which reminds me of a great talk by Gary Bernhardt titled “The Birth & Death of JavaScript”.

## Today I called Python from QML

I always use QML with Qt Quick for GUI programming. It’s incredible both for prototyping and larger applications. I find it easy to express myself in QML, because it is so flexible. It’s declarative. You can bind a button’s position to the value of a slider in just one line. Not that you’d ever want to do that, but it shows how easy it is to connect objects together. I really wish the web was written in QML and not HTML.

Typically, I’m working with both QML and C++. I write high performance and visualization code in C++ and define GUI elements in QML. But today I wanted to use QML and Python together because I’m working on an experiment browser for our lab. Not an experimental browser, but an application that lists all experiments and makes it easy to export data for further analysis.

We have decided to organize the experiments as HDF5 files. To read these files I want to use the Python HDF5 package. I could use the C++ HDF5 library, but using Python should hopefully make the application more easily maintainable in the future, also for non-C++ coders in our lab. To do this, I figured there were two possibilities: PyQt and PyOtherSide.

In brief (and a bit simplified), PyQt calls Qt code from Python, while PyOtherSide calls Python code from QML. The difference isn’t really that big, so it just boils down to where the business logic resides. I figured that PyOtherSide would be the better option for us, because it allows everyone to help out with the Python code without learning anything about Qt. PyQt would on the other hand require everyone to have at least some understanding of the Qt framework to make changes in the code. More of the business logic will have to take place in QML, though.

PyOtherSide is really simple in use. You just define a Python object in QML and this loads modules and files from the Qt resource file (qrc). Once loaded, Python functions can be called from QML and their results are automatically converted from Python types to Qt types:

import io.thp.pyotherside 1.3

Python {

return
}
}

function parseData(result) {
for(var i in result) {
var element = result[i]
tableModel.append(element)
}
}

Component.onCompleted: {
})
}
}

As you can see, all Python calls are asynchronous. You can also make synchronous Python calls, but this is not recommended because it could cause the QML GUI to stall while waiting for the results.

The Python code in this case is just a simple function that returns a dictionary:

def read_experiments():
return {{"experimenter": "test",
"date": "2015-10-26"},
{"experimenter": "test2",
"date": "2015-10-25"}}


Now I just need to make this code read and parse real HDF5 files.

## Recent commit causes Qt3D to fail compilation

A recent commit pushed to the Qt3D repository breaks building Qt3D against Qt version 5.2 and older with the following error message:

painting\qglpainter.cpp:2348:29: error: 'class QOpenGLFunctions' has no member named 'glReadPixels'

Thankfully, rolling back to an older commit still works. Just supply the following command after pulling the Qt3D repository:

git checkout d3338a9 -b older_version

This should enable you to compile Qt3D for Qt5.2 and earlier.

## New project structure for projects in Qt Creator with unit tests

Note: This is a new version of an earlier post, with a revised project structure.

Note 2: See this post for the same project structure using the even better Catch testing framework.

This post assumes that you are using a C++ testing framework such as UnitTest++. See this earlier post for some information on installing UnitTest++ on Ubuntu.

To get the most out UnitTest++ it is a good idea to integrate its output into the Qt Creator IDE. The way I have set this up in Qt Creator is with subprojects. One for the main project, which again is split into the app itself and a library, and one for the tests. In addition, I have a helper project file, named defaults.pri. The structure of the project is like this:

MyProject
├─ MyProject.pro
├─ defaults.pri
├─ app/
│  ├─ app.pro
│  └─ main.cpp
├─ src/
│  ├─ src.pro
│  └─ myclass.cpp
└─ tests/
├─ tests.pro
└─ main.cpp

An example project using this code structure has been posted on Github by Filip Sund. (Thanks to Filip for doing this!)

The main project file, MyProject.pro will now be based on a subdirs template, and may look like this:

TEMPLATE = subdirs
CONFIG+=ordered
SUBDIRS = \
src \
app \
tests
app.depends = src
tests.depends = src

The app.depends and tests.depends statements makes sure that the src project is compiled before the application and tests, because the src directory contains the library that will be used by both the app and the tests.

(Thanks to Will for noting that this works better for parallel builds with make -j 8 than my previous version only using CONFIG+=ordered. We should keep CONFIG+=ordered in there still, though, because depends doesn’t affect the order when using make install).

## defaults.pri

Each of the other .pro files will include defaults.pri to have all the headers available. defaults.pri contains the following:

INCLUDEPATH += $$PWD/src SRC_DIR =$$PWD

If the library, main program and tests use common libraries, it is very useful to have the defaults.pri define these dependencies too.

## ./src

In the src folder, I have myclass.cpp, which is the class that I want to use and test. The src.pro needs to compile to a library, so that it may be used both by app and tests, and could look something like this:

include(../defaults.pri)
CONFIG -= qt

TARGET = myapp
TEMPLATE = lib

SOURCES += myclass.cpp
HEADERS += myclass.h

What this class does is not so interesting, it could be anything. A simple example could be this header file:

#ifndef MYCLASS_H
#define MYCLASS_H

class MyClass {
public:
};

#endif // MYCLASS_H

With this accompaning source file:

#include "myclass.h"

double MyClass::addition(double a, double b) {
return a * b;
}

## ./app

I only have a main.cpp file in app, because the app is basically just something that uses everything in the src folder. It will depend on the shared compiled library from src, and app.pro would look something like this:

include(../defaults.pri)

TEMPLATE = app
CONFIG += console
CONFIG -= app_bundle
CONFIG -= qt

SOURCES += main.cpp

LIBS += -L../src -lmyapp

The main.cpp file could be a simple program that uses MyClass:

#include <myclass.h>
#include <iostream>

using namespace std;

int main()
{
return 0;
}

## ./tests

In the tests folder I have simply added a main.cpp which will run the tests. Then tests.pro has the following contents:

include(../defaults.pri)
TEMPLATE = app

CONFIG   += console
CONFIG   -= app_bundle
CONFIG   -= qt

SOURCES += main.cpp

LIBS += -lunittest++ -L../src -lmyapp

Which now links to the myapp library in addition to the unit tests.

The main.cpp in tests file which could contain the following, if we were to use UnitTest++ as our testing library:

#include <unittest++/UnitTest++.h>
#include <myclass.h>

TEST(MyMath) {
MyClass my;
}

int main()
{
return UnitTest::RunAllTests();
}

This test will fail because my implementation of MyClass::addition is completely wrong:

class MyClass {
public:
double addition(double a, double b) {
return a * b;
}
};

Note that I’m including MyClass by through <myclass.h> which is possible because of the INCLUDEPATH variable in defaults.pri.

This is hopefully all you’ll need to define a project that compiles a library, as well as tests and an application using the library.

Edit: In QtCreator 3.1, the method used here (using resource files) will become the default way to perform deployment on all platforms. I.e., this guide is only useful if you have created your project using QtCreator < 3.1.

If your Qt application is taking too long to load on Android, it is likely because accessing asset-files is really slow on some devices. I just tested an application with about 12 QML-files and it took 40(!) seconds to load. This appears to have been an issue for quite some time, and although efforts have been made to reduce the problem, my app still loads too slow on my Asus Transformer TF300T. In Qt and Android’s defence, this device is known for its extremely slow eMMC storage.

The cure in my case was to use a Qt Resource Collection file for all my QML assets. In Qt Creator, this is as simple as clicking  File > New File or Project > Qt > Qt Resource file and then add a prefix “/” before adding all files from each QML folder. After creating and setting up the .qrc-file, I changed the URL in my main-function to “qrc:/qml/myproject/main.qml”.

This worked flawlessly, and the load time dropped from 40 seconds to less than a second!

You should of course be careful with what you add to your resource file. All listed files will be compiled into your executable, so if there are files you don’t want loaded right away, you might want to keep them outside your .qrc file.

The only downside with this approach is adding all new QML files to the resource file. This  is a bit tedious in the long run, so I suppose I will automate it with a script.

## Copying data files to the build directory with qmake

Sometimes you want to copy some data files to the same folder as your final executable when you are building with qmake in a different directory than your source code. This is what Qt Creator does by default, so this is quite often needed.

Just add the following to your .pro file to copy the folder “data” from the source directory to the build directory whenever you build the source code:

copydata.commands = $(COPY_DIR) $$PWD/data$$OUT_PWD first.depends =$(first) copydata
export(first.depends)
export(copydata.commands)
QMAKE_EXTRA_TARGETS += first copydata

The meaning of the different names in the above are as follows:

• \$(COPY_DIR) – Holds a platform-dependent copy-command that makes sure to copy recursively.
• $$PWD – Holds the name of the source code directory, where your .pro file resides. •$$OUT_PWD – Holds the name of the build directory. Note that this may not work if you are running qmake and make in the same directory as your source code, as you will be copying into the same folder as you copy from.
• copydata – Just a name we choose. You can replace this with whatever you want.
• first – This is a build step that already exists in the Makefile generated by qmake. We attach the copy build step to this build step.
• export – Makes the variable in the argument of the function available in a global scope.
• QMAKE_EXTRA_TARGETS – Tells qmake that you want to build more targets than the ones it adds by default to the Makefile.

This method is a bit messy, and I wish the Qt developers would make it easier to do this, but it works. It also ensures that the data is always copied, ensuring that any changes in the data folder are pushed to the build folder.

## Multiple billboards in Qt3D

While Qt3D has a transform for drawing billboards, it is not very useful if you are drawing thousands of particles. This is because the transform can only be applied to one object at the time, and using thousands of objects is never a good idea in neither OpenGL or Qt. This is simply because the overhead of each object is very large, both in terms of memory and performance.

The solution I found was to write my own custom class that inherits QQuickItem3D and is capable of drawing a list of 3D points as billboards by directly turning each sprite towards the camera at every draw call. This is not very efficient, but it works and is way faster than the options: Drawing spheres or creating one billboard per particle. The ideal option is of course to use geometry shaders, but they are not easily available in Qt3D yet. Most likely because its a new feature in OpenGL 3.2.

The class is available on GitHub and the resulting output is shown in this screenshot: