When I started working in Zivid, a Norwegian company that makes industrial 3D cameras, one of my first tasks was to port our compute pipeline from OpenCL to Halide. I had never heard about Halide at the time. It was one of many options we considered when looking for a replacement for OpenCL. Halide stood out because it allowed us to write our algorithms in Python, optimize the scheduling separately from the algorithms, and compile functions that could be called from C++.

Most of our research code was, and still is, written in high-level languages like Python and Matlab. Being able to cut down the time it takes to go from a research prototype to the final product has been of huge importance in our work.

However, Halide is different from other Python libraries. It is a domain-specific-language (DSL) with a syntax that differs from many of the other libraries in the Python ecosystem. Halide also has its own set of quirks and features that need to be understood to get the most out of it. We have had to learn many things the hard way when working with Halide.

That is why I wanted to write this book. While Halide has great documentation and tutorials, there are few resources available with practical examples that contain the type of learnings we have made along the way. Halide's community is great with a core team of responsive developers and users that are happy to share their thoughts. This book is my way of giving back to that community.

Halide is an interesting library. It is not particularly famous or well-known. Not many books are written about it. Yet, it has powered and still is powering some of the most influentatial image processing software in the world, including Adobe's Photoshp, Google's YouTube, and Pixel phones.

Calling it a library also does not really do it justice. It is a language. Yet, it is so important to emphasize that it is embedded in Python and C++. You will not need to learn an entire new language. Just some added syntax on top of a language you already know.

What is Halide?¶

Halide is a language designed for image processing. It runs on top of Python or C++ and compiles down to a number of backends.

What sets Halide apart from other libraries and languages in image processing is that it decouples algorithms from scheduling. What that means is that you can define the computations in your algorithm without worrying about order of operations and memory layout, which is known as the schedule.

This turns out to be incredibly powerful, as it allows you to concern yourself with the algorithms first, and then address performance later. Further, you can rest assured that the resulting algorithm stays the same, while you play around with optimizing the order of computations for performance.

Format of this book¶

This book is written using a mix of Markdown and Python files. It is an executable book, meaning that the contents are generated from running the Python code included in the book. If you are familiar with Jupyter Notebooks, the concept is similar. Just imagine that this book was written entirely from notebooks and converted to the format you are reading it in now, whether that is a PDF or online HTML files.

Whenever you encounter a code cell like this, you can also see the output below it and know that it corresponds to what you would get if you were to run the same code in Python yourself:

And since the book is generated from executing this code, you can rest assured that it is up-to-date.

It is also an interactive book, which means that if you are reading it online, you will be able to interact with several of the examples.

Structure of this book¶

The book is layed out as a number of examples, progressing from a comfortable introduction to the basics of Halide, through more advanced use-cases to complex programs. Personally, I always find it easier to learn with a problem-based learning and a motivating use-case at hand.

By the end of this book, we will have made an image-processing pipeline that can enhance images for consumption both by humans and machines.

The book is still a work-in-progress and will be expanded over time. Even though it is currently quite short, I hope you find it useful.