The world’s leading publication for data science, AI, and ML professionals.

Raspberry Pi and OpenVino

With the Intel Compute Stick

DOING DATA SCIENCE FROM SCRATCH TASK BY TASK

Image by the author - showing the authors personal Raspberry Pi 4b 8G with Intel support
Image by the author – showing the authors personal Raspberry Pi 4b 8G with Intel support

If you have been reading my column here on Towards Data Science, you will know that I am on a mission. I wanted to count the number of cars passing my house using Computer Vision and Motion Detection. This article provides a small update on Computer Vision using the Intel Neural Compute Stick and the OpenVino library.

As a catch-up, I previously built a camera and then explained the need to tune the motion detection device(s). My earlier posts described the camera build and early scripts to review the data and make sense of passing traffic. We had even made a very early chart to plot motion events. Last time I sent 192 images forward through the Yolo model using the Raspberry Pi ARM CPU. There was a lot of heat, and things took a long time. In fact, it took 10 seconds per image. Worse still, I had to slow everything down to only do a shot in 15 seconds. The system over-heated getting to 85% degrees, and I closed, for XMAS, with wanting to run the Neural Compute Stick with that workload to see if things would be any better. My love for Raspberry Pi had started to wear off, and I was feeling a little sad. Let’s get on with updating my code to use the Intel co-processor and just see what happens!

Updating my class and re doing the workload

I mentioned I had the Neural Compute Stick in my last article, I therefore went ahead and refactored my class to exploit OpenVino. That would allow me to transfer the CPU workload to a specialist Inference Engine. Since I had committed myself to report back on the effort here is my summary of the exercise.

There is an excellent tutorial on the whole topic over at pyimagesearch.com and plenty written about OpenCV and OpenVINO already such that I won’t need to get into the details here.

OpenVINO, OpenCV, and Movidius NCS on the Raspberry Pi – PyImageSearch

Updating my code

The value of using the Object Orientated approach is that the code gets nicely segregated out and is therefore much easier to maintain and update. I only really needed to add a single line of code to my init function. After I load the ‘net’ from the disk files, I can update the Object to set my preferred device as the Intel stick. How cool is that!

self.net.setPreferableTarget(cv2.dnn.DNN_TARGET_MYRIAD)

Naturally, there are all sorts of problems when we take this simple approach. What if the Neural Compute Stick isn’t plugged in? Rather than default to the Raspberry Pi CPU I prefer the script to abend! Obviously, for production workloads, you’d really want to write more robust code! The last run, 192 photos, took 15 seconds per image, and produced a very hot Raspberry Pi, so how did we do?

Using the Intel Neural Compute stick

I couldn’t be happier as it turns out. The co-processor cost me about 100.00 dollars, far less than an Nvidia Titan GPU and is much easier to set-up. I couldn’t be happier, really.

Intel Neural Compute Stick 2

So I ran my code once again

my = myYolo()
my.run()

This time I removed a 5-second delay to allow things to cool down. I can tell you that the compute stick is a Turbocharger (Supercharger) add-on to **** the _Raspberry P_i for Deep Neural Network inference workloads. Let us examine the results.

Image from the authors Excel file
Image from the authors Excel file

Image 0 – took 29 seconds, and that might seem like a lot. But remember that we have the overhead of transferring our model to the Neural Compute stick on the first forward pass. Image 1 took a mere .58, so just half a second to perform a forward pass on the neural network. If we exclude image 0, as an outlier, and get the average of the remaining 191 images, the average inference time is .57 or half a second.

The Intel device performs object detection in .57 seconds whereas the Arm-based CPU for Raspberry Pi 4b took at least 9.5 seconds. Isn’t that incredible! But wait! there is more. Since we are offloading the compute-intensive workload to the Intel device, there is nothing much for the CPU to do. The CPU must mostly handle a bit of I/O between the SSD storage drive, cache memory, and the Neural stick. Therefore there is absolutely no heat produced. We went from >80degrees to a more or less constant 45degrees. That really is amazing. The Intel stick gets warm to the touch, but there is no hum or any noises at all.

My love of the Raspberry Pi is now restored, and I am delighted.

Next steps

Now it makes no difference, to the accuracy, if you use the Intel Neural Compute stick or the Arm-based CPU. Things just move slower or faster, but the object detection is the system. I reported back on doing Computer Vision operations on the Intel Stick, but I still have my investigation into 100 pictures where Yolo didn’t find an object to complete! Is the data pointing to a need to train Yolo on an Irish data set? Exciting, please do come back! I can now run Yolo in a reasonable time and pass many more photos through the model.


Related Articles