Halide supports the Python buffer protocol, so we can use any library that supports loading images into such a buffer. Most libraries load their images into a NumPy array, which also supports the buffer protocol.

Halide also provides a wrapper around the imageio library, which further ensures the image data is layed out the right way. For simplicity, we will use this throughout this book.

In [1]:

import matplotlib.pyplot as plt
import halide.imageio
import halide as hl
from halide_book.paths import Paths

In the code base of this book, there are a few example images that we can use. Paths.static is a constant containing the path to which we have stored the images. If you want to load a file of your own, simply pass the path of the file to the halide.imageio.imread function instead.

Here we load one of the example images:

In [2]:

image_array = halide.imageio.imread(Paths.static / "stuff.png")

Note that while the image is loaded with the layout required by Halide, the same layout is not what for instance imshow expects. To convert to the right layout for imshow, we need to call the copy_to_interleaved function:

In [3]:

image = halide.imageio.copy_to_interleaved(image_array)
plt.imshow(image)
plt.show()

No description has been provided for this image

We will use the above functions whenever we load and visualize images in the coming chapters.

Next, we store this array in a Halide Buffer object, which will allow us to use it in Func definitions.

In [4]:

image_buffer = hl.Buffer(image_array)

Next, we can define a Func that performs an operation on the loaded image. As an example, we will brighten it by multiplying every value in the image by two.

This time, we define the coordinate system using x and y as before, but we also need a third dimension to represent the red, green and blue color channels. We will call this dimension c for "channel".

In [5]:

x = hl.Var("x")
y = hl.Var("y")
c = hl.Var("c")

Next, we define the brighten function

In [6]:

brighten = hl.Func("brighten")
brighten[x, y, c] = 2 * image_buffer[x, y, c]

Note how we can now use the coordinates on the left hand side of the definition of brighten to essentially look up the pixels in the image we loaded.

Let us check out the result.

In [7]:

result = brighten.realize(
    [
        image_buffer.width(),
        image_buffer.height(),
        image_buffer.channels(),
    ]
)
plt.imshow(halide.imageio.copy_to_interleaved(result))
plt.show()

Yuck! That does not look good. What went wrong?

The problem is that the loaded image contains only 8-bit values:

In [8]:

print(image_buffer.type())

uint8

The definition of brighten then also results in a Func containing only 8-bit values.

In [9]:

print(brighten.type())

uint8

This means that each value is between 0 and 255 and that when we perform math on these values, we are at the risk of getting overflow and underflow in our results. This means that for instance 2 * 150 does not become 300, but instead becomes 300 - 255 = 45. That is why we get those weird color artifacts in the above image.

Before we perform any modification to the image, we should convert its data to float values. Next, we need to convert it back to 8-bit unsigned integer values before plotting.

This can be done with hl.cast or one of the conversion functions, such as hl.f32, hl.f64, hl.u8. In this book, we will mostly use the conversion functions.

Let us first define the image as float values:

In [10]:

image_f32 = hl.f32(image_buffer[x, y, c])

Next, we implement our brighten function again:

In [11]:

brightened = 2 * image_f32

We now need to clamp the values and convert them back to 8-bit integer values.

In [12]:

brighten_u8 = hl.u8(hl.clamp(brightened, 0, 255))

Finally, we put the expression back into a Func:

In [13]:

brighten = hl.Func("brighten")
brighten[x, y, c] = brighten_u8

When we now realize the result and plot the image, it should look better.

In [14]:

result = brighten.realize(
    [
        image_buffer.width(),
        image_buffer.height(),
        image_buffer.channels(),
    ]
)
plt.imshow(halide.imageio.copy_to_interleaved(result))
plt.show()

That looks much better!

We are now ready to perform more advanced operations on images.