One of the most popular development languages at the moment is Python, it stands to reason that we want to be able to use Python in our Zynq and Zynq MPSoC applications.

Image processing using Pynq on the Ultra96 is a great use case

This is where the Pynq framework comes in. Pynq enables developers to use Python to leverage the programmable logic provided by the Zynq and Zynq MPSoC. While we have looked at the Pynq board in the past (installments 155 to 161 & 237), this was with respect to the Pynq Z1.

Recently, Avnet built and released the Pynq framework targeting the Ultra96. This is an exciting development as it further opens up the Ultra96 to developers who want to benefit from the acceleration provided by programmable logic, without the requirement to be a programmable logic specialist.

The Pynq framework enables developers to use the programmable logic thanks to a number of hardware libraries called overlays. Overlays are open source programmable logic designs which can be loaded as needed to provide acceleration in the execution of a Python program.

Of course, if an overlay does not exist and you have the skills, then you can create your own overlay and share it with the community.

Along with overlays another key element of the Pynq framework is the use of Jupyter notebooks. These notebooks allow us to develop, run and document our Python application.

To use the Pynq framework on the Ultra96, we need to have an updated Linux image which includes the framework. There are two ways we can obtain this:

Download the ISO image from Zedboard.org

Create our own image from the sources available here .

For this blog I used the pre-generated image.

Once the image is downloaded, the next step is to write the image to an SD card just like we do with any other Linux Image.

With that completed we are in a position to start working with the Ultra96 and Pynq.

If you have worked with the Pynq previously you will know we need to use a network connection to access the Jupyter notebooks.

To connect to the Ultra96, we have three options:

Use a USB Ethernet connection over the MicroUSB (upstream USB connection).

Connect via WiFi.

Use the Ultra96 as a single-board computer and connect a monitor, keyboard and mouse.

The fastest and easiest way to get up and running is to use the USB Ethernet connection. We can then use this to connect the WiFi if we want to access the internet and download available overlays from the Pynq community.

With the USB cable connected, navigate to 192.168.3.1, this will open the Pynq login page. At the password prompt, log in with the password “xilinx.”

Pynq log-in page

Logging in will present you with a number of folders, under common you will find a number of example note books. Getting started provides some basic information on working with the Pynq. While the Sensors96B is for working with the Grove starter kit.

Initial files

The next step is to get the Ultra96 connected to WiFi so we can connect to the internet and download new overlays if desired.

To do this, we need to run the wifi.ipynb which can be found under the common folder.

This Jupyter notebook will enable us to enter in the SSID and network password for the network we wish to connect to.

Running the WIFI.ipynb notebook

Now we have the initial setup for the Pynq on Ultra96 working and connected to the internet, we can add overlays if we desire from the community of developers.

One very useful repository is the Pynq computer vision repository. This contains a range of overlays for both the Pynq Z1 and Ultra96 which accelerate image processing functions. For the Ultra96, three overlays are provided which provide PL acceleration of:

2D filter, dilute and absolute difference

Remap and stereo LBM

Optical flow

These are very useful overlays for many image processing applications, especially the 2D filter overlay as it lets us create and apply custom kernel filters.

To add in the Pynq computer vision overlays, we need to connect to the Ultra96 using SSH (Username Xilinx, Password Xilinx). Once we have connected over SSH, we can run the command below to download and install the overlays and associated Jupyter notebook examples.

sudo pip3 install --upgrade git+https://github.com/Xilinx/PYNQ-ComputerVision.git

SSH download

Once installation is complete, upon returning to the Pynq homepage you will notice a new folder which includes the PynqOpenCV overlays.

Updated homepage following the Pynq OpenCV installation

Within the Pynq OpenCV folder you will see several examples — these examples run the same algorithm in both the software and hardware. Of course, this provides a very good demonstration as to the power and ease with which we can work with Pynq.

Pynq OpenCV examples

To run these examples I needed to connect a USB 3 camera to the Ultra96. This example uses the e-con systems 3.4 Mega Pixel low light camera.

Pynq connected to USB3 Camera

To demonstrate the acceleration provided by the Pynq OpenCV overlays I ran both the 2D filter and optical flow examples.

Rather helpfully, both examples provide plots of performance in SW and HW for comparison.

These results can be seen below, as would be expected both demonstrated significant improvements in frame rate and execution time when using the Pynq OpenCV overlays.

Comparison between SW and HW for 2D filter example

Comparison between SW and HW for optical flow example

In my view, adding the Pynq framework to the Ultra96 really opens up the platform.

In a future blog, we will look at how we can create our own overlay.

Find out more about Pynq on Ultra96 here.

See My FPGA / SoC Projects: Adam Taylor on Hackster.io

Get the Code: ATaylorCEngFIET (Adam Taylor)

Access the MicroZed Chronicles Archives with over 250 articles on the Zynq / Zynq MpSoC updated weekly at MicroZed Chronicles.