Today I called Python from QML

I always use QML with Qt Quick for GUI programming. It’s incredible both for prototyping and larger applications. I find it easy to express myself in QML, because it is so flexible. It’s declarative. You can bind a button’s position to the value of a slider in just one line. Not that you’d ever want to do that, but it shows how easy it is to connect objects together. I really wish the web was written in QML and not HTML.

Typically, I’m working with both QML and C++. I write high performance and visualization code in C++ and define GUI elements in QML. But today I wanted to use QML and Python together because I’m working on an experiment browser for our lab. Not an experimental browser, but an application that lists all experiments and makes it easy to export data for further analysis.

We have decided to organize the experiments as HDF5 files. To read these files I want to use the Python HDF5 package. I could use the C++ HDF5 library, but using Python should hopefully make the application more easily maintainable in the future, also for non-C++ coders in our lab. To do this, I figured there were two possibilities: PyQt and PyOtherSide.

In brief (and a bit simplified), PyQt calls Qt code from Python, while PyOtherSide calls Python code from QML. The difference isn’t really that big, so it just boils down to where the business logic resides. I figured that PyOtherSide would be the better option for us, because it allows everyone to help out with the Python code without learning anything about Qt. PyQt would on the other hand require everyone to have at least some understanding of the Qt framework to make changes in the code. More of the business logic will have to take place in QML, though.

PyOtherSide is really simple in use. You just define a Python object in QML and this loads modules and files from the Qt resource file (qrc). Once loaded, Python functions can be called from QML and their results are automatically converted from Python types to Qt types:

import io.thp.pyotherside 1.3

Python {
    property bool ready: false
    
    function loadData() {
        if(!ready) {
            return
        }
        call("hdf5_loader.read_experiments", [], parseData)
    }

    function parseData(result) {
        for(var i in result) {
            var element = result[i]
            tableModel.append(element)
        }
    }
    
    Component.onCompleted: {
        addImportPath(Qt.resolvedUrl("."))
        importModule("hdf5_loader", function() {
            ready = true
        })
    }
}

As you can see, all Python calls are asynchronous. You can also make synchronous Python calls, but this is not recommended because it could cause the QML GUI to stall while waiting for the results.

The Python code in this case is just a simple function that returns a dictionary:

def read_experiments():
    return {{"experimenter": "test", 
             "date": "2015-10-26"}, 
            {"experimenter": "test2", 
             "date": "2015-10-25"}}

Now I just need to make this code read and parse real HDF5 files.

Straight from the source: NEURON’s incredible backwards compatibility

NEURON is a neural simulator, but it’s not just another neural network application. NEURON has been around much longer than your fancy deep learning software. It was first released in 1994, although some of it’s early code and concepts seem to stem from the 70’s and 80’s. In fact, NEURON is not a machine learning library at all. It simulates real neurons. It uses real morphologies and ion channel densities to reproduce experiments. Some labs even use machine learning algorithms to fit the parameters of their NEURON models. NEURON is one of the workhorses of the computational branch of the Human Brain Project and the core engine in the Blue Brain Project.

But the age of NEURON is starting to show in its source code. Because some things are a bit awkward when working with this simulator I decided to peek at its sources. Just to see what this old engine looks like on the inside. And it turned out to be an interesting journey. Deep inside NEURON’s plotting library, I found this:

#if VT125
case VT:
vtplot(mode, x, y);
break;
#endif

Now, I didn’t know what VT125 was when I first saw this, but a quick search on the web reminded me that I’m still a young software developer. I present to you, the VT100:

VT100 terminal
I couldn’t find a good picture of the VT125. I apologize if it looks way more modern than its older cousin.

Now that’s what I call backwards compatibility!

Some of you may want to lecture me about how modern terminals actually emulate VT100s and that this code might be in there to allow some fancy plotting inside xterm or its siblings. But I would argue that there are other parts of this code that give away its impressive attempt at supporting older platforms. Such as the rest of the above switch statement:

		switch (graphdev)
		{
		case SSUN:
#if SUNCORE
			hoc_sunplot(&text, mode, x, y);
			break;
#else
#if NRNOC_X11
			hoc_x11plot(mode,x,y);
			break;
#else
#if NeXTstep
			break;
		case NX:
			hoc_NeXTplot(mode,x,y);
			break;
#endif
#endif
#endif
#if TEK
		case ADM:
		case SEL:
		case TEK4014:
			tplot(mode, x, y);
			break;
#endif
#if VT125
		case VT:
			vtplot(mode, x, y);
			break;
#endif
		}
#endif

Now, we all love nested preprocessor if-statements inside switch blocks, but let’s look aside from that and at what’s supported here.

There’s the NRNOC_X11, which I believe introduces the only part of this block that might actually be executed nowadays. In addition we have SUNCORE, which might be Solaris, but I’d bet this supports something that existed long before Oracle acquired Sun back in 2010. There’s TEK, which may refer to something like the Tektronix 4010:

Tektronix_4014
This baby was made in 1972, but it still runs NEURON’s plotting functions. Just download the 5 MB zip-file of NEURON’s source code and … oh, wait.

And then there’s NeXTstep, which Apple acquired in 1997 and used to replace its own Mac OS with Mac OS X. Considering that NeXTstep made its last official release in 1995, I think its fair to say that this piece of code was written in the previous century. It could, of course, just be that no one has deared to search-replace NeXTstep with OS X, but I doubt it.

I should finish this with a final caveat: It could be that this code isn’t at all in use by NEURON anymore. After all, I found all of the above in the plot.c file in the src/oc folder of NEURON’s source code. The above could be remainders of NEURON’s interpreter language, hoc, which was originally made as a demonstration of how to use the parser generator Yacc in the The Unix Programming Environment book. As far as I know, NEURON is the only software out there that’s still using hoc, but that’s a story for another day.

Image Credits: Wikipedia User:ClickRick, Wikipedia User:Rees11.

Using Qt3D today

Qt3D is an amazing library for Qt that gives you the ability to render your own 3D stuff together with your existing widget or QML based GUI. The library was started by the Qt developers a few years back, but has not yet been released with the official Qt SDK. It was announced that it would be bundled with Qt5.0, but because of the state of the library at the time, its release was postponed. Even so, the library is still very mature and appears to work very well in my opinion. So I would urge you to test it out.

Simple install on Ubuntu

To use the library you have a couple of different choices. If you are running Ubuntu, you may just run the following commands:

sudo apt-get install qt3d5-dev qtdeclarative5-qt3d-plugin

And you should be good to go. Look for examples in the Qt3D documentation.

Note that Ubuntu currently doesn’t have QtQuick.Controls available, so you will have to follow the instructions below to combine Qt3D with QtQuick.Controls.

Install Qt3D into the newest Qt release

Currently Qt5.1.1 is the newest stable release of Qt and is the version I’d recommend that you use with Qt3D.

  1. Download and install Qt5.1.1.
  2. Clone Qt3D from Gitorious:
    git clone git://gitorious.org/qt/qt3d.git qt3d
  3. Open QtCreator from your new Qt installation.
  4. Open the qt3d.pro file from the cloned repository and configure it to use the platforms on which you want to use Qt3D (desktop, Android, etc.).
  5. Build for each platform you want to use. Set the build to Release instead of Debug mode while building. You should receive about one hundred warnings if you are building for Android. As always, ignore these.
  6. Open the build folder in a terminal and run
    make install

    This will not install to your system, but to your Qt installation, so that it gets the ability to do awesome 3D stuff.

You should now have Qt3D installed in your Qt installation and be ready to run projects that are using Qt3D.

Monitoring your unit tests without lifting a finger

I love unit testing. First of all, I think it is a good idea to test separate units of the code, but after doing so for some time, I’ve come to realize that unit tests are great for managing the software development cycle too. It all boils down to the idea that you should write tests before you write your code.

Now, this is something that I and others apparently struggle a lot with. How do you write a test for some code that doesn’t even exist yet? Even worse, how do you write a test for a piece of software that you’re not yet sure how will be used?

In computational physics, this problem arises often because we are writing code at the same time as we are trying to understand the physics, mathematics and algorithms at hand. And this is a good thing. You might want to think that one should structure all code before it is written, but this is generally a bad approach in computational physics. Especially if you’re working on something new. The reason is that you will often understand the problem and algorithms better while developing, rather than just reading about them and trying to analyze them blindly.

Keeping the tests and code healthy

But enough with the talk, let’s just assume that you are convinced that you should (or have to) implement some unit tests. At one point you are likely to be in a position where you find it tiresome to have to go into that folder where the tests are defined and run them manually. This is where Jenkins comes in to play.

Continue reading Monitoring your unit tests without lifting a finger

Using the same script on installs with different EPD versions

In the newest version of Enthought’s Python Distribution (EPD) on Ubuntu, the plotting package has been moved from enthought.mayavi.mlab to the shorter and more general mayavi.mlab. This does however mean that if you, like me, need to work with different versions of EPD on multiple systems, will experience the following error from time to time:

ImportError: No module named enthought.mayavi.mlab

Now, to avoid switching the import statement every time you switch systems, you can make Python check if one of the versions is installed during import. If it is not, we’ll tell it to try the other. This is done in this simple command:

try:
    from enthought.mayavi.mlab import *
except ImportError:
    from mayavi.mlab import *

Just replace any other similar import statements the same way and your code should once again be working across all your installations.

Trafikanten on Maemo

Update: The package was pushed to Extras testing on 2nd of October. It was delayed because Maemo’s package repository would not import the new package due to some trouble with the version numbering.

There has been missing a native application to access information from Trafikanten on the Nokia N900. Instead it has been necessary to use the mobile version of their websites or maybe even the full version. Even though this is a pretty quick and workable solution, there are several benefits of having a native application available. One that becomes very obvious is the use of GPS to find all nearest bus stops or train stations.

I decided to give it a try in June this year and started by releasing some early versions of what is now named “Journey Planner for Norway” (which is “Reiseplanlegger” in Norwegian). During the summer of 2010 I have been trying to improve the application over several iterations, and feel that it has now come to a point where it is usable and stable enough to get the infamous “Version 1.0” attached to it.

The application is now finding its way through to the Extras-devel catalogue for Maemo and will be pushed upwards through Extras-testing before it hopefully hits Extras within two or three weeks. In the meantime you might either be a bit careless and enable Extras-devel to use the bleeding edge 1.0 version or head over to Maemo Downloads to fetch the beta which will be automatically updated to 1.0 whenever it is ready.

Below you can check out some more screenshots of the application in action:

The application is developed in C++ using Qt with the Qt Mobility libraries. It should not be too hard to port it to Symbian as well, so I will give that a shot in the future. Symbian users does however already have an option to use the Java application “Trafikanten Sanntid” made by Håvard Tegelsrud.

And of course, you can check out the source code here. The application is licensed with the GNU GPLv3 license.

Fixing “undefined reference to `vtable for …”

These annoying errors have been haunting me the last couple of days, so I figured I should share the most common reason for their occurrence. That is in my projects at least.

This error is caused because the linker in gcc is unable to find the functions you have defined in your headers in your actual code. So if you have a header which looks like this:

#ifndef MESH_H
#define MESH_H

class Mesh
{
public:
    Mesh();
    ~Mesh();
    virtual void draw();
};

#endif // MESH_H

You must at least have these functions defined in your .cpp file:

#include "mesh.h"

Mesh::Mesh() {
}

Mesh::~Mesh() {
}

void Mesh::draw() {
}

After this, make sure you clean your compile environment to make sure no object files are being misinterpreted by the compiler. If you are using Qt or a project with a Makefile, you could just run these three commands (the first only applies to Qt projects).

qmake
make clean
make

Should you still have trouble, make sure that qmake is actually generating your .moc files for any objects that need them. Sometimes it might even be necessary to empty the build directory completely yourself to make sure there are no files left behind that are not cleaned by make clean.