OpenCV + CUDA + AWS EC2 + (No More Tears)
Step by step instructions to bind OpenCV libraries with CUDA drivers to enable GPU processing on OpenCV codes.
By default, there is no need to enable OpenCV with CUDA for GPU processing, but during production, when you have heavy OpenCV manipulations to do on image/video files, we can make use of the OpenCV CUDA library to make those operations to run on GPU rather than CPU and it saves a lot of time.
It was not easy as it is said to connect the OpenCV library to enable it with CUDA, I had to go through a painful process for a week to establish the connection properly, also its both time & money consuming process. So this time I want to record the overall process for my future, as well as for others.
For the demonstration, I am renting an EC2 instance with a p3.8xlarge instance in the AWS, which has 4 Nvidia GPUs.
So if you need any help in starting an EC2 instance for the first time, you can refer to my previous post on Step by Step Creation of an EC2 Instance in AWS and Access it via Putty & WinSCP and during the process select the GPU instance you require.
Now after ssh-ing into the instance, before we get into the process we need to install a lot of packages to make the environment ready.
Note: I have consolidated all the commands I ran from start to end and added them at the bottom. If you are more curious find them here in this link and follow along.
Run the below commands one after another on your instance and also I have attested the screenshots to compare the outputs against mine.
All the screenshots used hereafter are sourced by the author.
Step 1: Install OpenCV Dependencies, Nvidia CUDA driver, CUDA toolkit.
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install build-essential cmake unzip pkg-config
sudo apt-get install gcc-6 g++-6
sudo apt-get install screen
sudo apt-get install libxmu-dev libxi-dev libglu1-mesa libglu1-mesa-devsudo apt-get install libjpeg-dev libpng-dev libtiff-dev
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-devsudo apt-get install libxvidcore-dev libx264-dev
sudo apt-get install libopenblas-dev libatlas-base-dev liblapack-dev gfortransudo apt-get install libhdf5-serial-dev
sudo apt-get install python3-dev python3-tk python-imaging-tk
sudo apt-get install libgtk-3-dev
sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt-get update
sudo apt-get install nvidia-driver-418
sudo reboot
After the installation, reboot the system. Once your system brings up, type nvidia-smi and it should give similar output, as you see in the below snippet, I am powered with 4 Tesla v100 GPUs.
mkdir installers
cd installers/
wget https://developer.nvidia.com/compute/cuda/10.0/Prod/local_installers/cuda_10.0.130_410.48_linuxmv cuda_10.0.130_410.48_linux cuda_10.0.130_410.48_linux.run
chmod +x cuda_10.0.130_410.48_linux.run
sudo ./cuda_10.0.130_410.48_linux.run --override
This is the most important step of all, which is not detailed in most of the articles I found online and it took more than a day to solve this puzzle.
After you read and accept the EULA agreement, there are some yes(y)/No(n) commands and fill it based on the below response.
Now after you successfully installed that, you will be provided the PATHs which need to be added to bashrc file and later source it.
sudo vim ~/.bashrc
source ~/.bashrc
nvcc -V
Note the commands under the #NVIDIA CUDA Toolkit and add them based on your paths that you’ve got in the previous command.
# NVIDIA CUDA Toolkit
export PATH=/usr/local/cuda-10.0/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-10.0/lib64:$LD_LIBRARY_PATH
Now source it, to save the changes and later install some dependencies.
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install build-essential cmake unzip pkg-config
sudo apt-get install libjpeg-dev libpng-dev libtiff-dev
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev
sudo apt-get install libgtk-3-dev
Step 2: Download OpenCV Source Code
Now we need to download OpenCV & OpenCV contrib and manually run their setup files. Here I am using versions 4.2.0 of the OpenCV package.
cd ~
wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.2.0.zipwget -O opencv.zip https://github.com/opencv/opencv/archive/4.2.0.zipunzip opencv.zip
unzip opencv_contrib.zip
mv opencv-4.2.0 opencv
mv opencv_contrib-4.2.0 opencv_contrib
Step 3: Configure Python Virtual Environment
Here we are using a virtual environment and will be building the bindings of OpenCV CUDA inside the environment hereafter.
In this part, I will use virtualenv and virtualenvwrapper as the virtual environment.
wget https://bootstrap.pypa.io/get-pip.py
sudo python3 get-pip.py
sudo pip install virtualenv virtualenvwrapper
sudo rm -rf ~/get-pip.py ~/.cache/pip
After installing these packages, you need to add these lines in ~/.bashrc in order to let bash load virtualenv and virtualenvwrapper each time when the terminal is up:
sudo vim ~/.bashrc
# Add them inside bashrc# virtualenv and virtualenv wrapper
export WORKON_HOME=$HOME/.virtualenvs
export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3
source /usr/local/bin/virtualenvwrapper.sh
Now after additions, source it. now runsource ~/.bashrc
# Create a virtual environment. The first step is to create a virtual environment:mkvirtualenv opencv_cuda -p python3
pip install numpy
nvidia-smicd ~/opencv
mkdir build
cd build
Here I am installing the Numpy library inside the virtual environment.
Step 4: Determine Your CUDA Architecture Version
to find the architecture runnvidia-smi and note down the name of the Nvidia GPU you are assigned to. In my case, I have Tesla v100
As an experienced CUDA programmer, determining the CUDA architecture version is a required practice because it lets the compiler generate more efficient code on your GPU. Furthermore, setting architecture params which does not include the architecture of your Nvidia GPU will let your program not working while executing.
We can use nvidia-smi to figure out what model of your Nvidia GPU is:
After you get the model of Nvidia GPU, you can find your CUDA Architecture using this page:
Scroll down to the Paragraph of “Your GPU Compute Capability”. As I’m using Nvidia Tesla v100, I will click on the “CUDA-Enabled Tesla Products” sections.
After examining it, I realize my Nvidia GPU architecture version is 7.0. As a reminder, your GPU architecture version may vary.
Once you got the GPU architecture version, leave a note of it because we will use it on the next step.
Step 5: Configure OpenCV with Nvidia GPU Support
OpenCV uses CMake to configure and generate the build. First of all, activate the opencv_cuda virtual environment, if it's not enabled before.
Also in the CMake commands, in the argument CUDA_ARCH_BIN=x.x, replace that with the compute capacity version you got in the previous step. Here I am using 7.0, kindly replace it with your version.
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D INSTALL_PYTHON_EXAMPLES=ON -D INSTALL_C_EXAMPLES=OFF -D OPENCV_ENABLE_NONFREE=ON -D WITH_CUDA=ON -D WITH_CUDNN=OFF -D OPENCV_DNN_CUDA=ON -D ENABLE_FAST_MATH=1 -D CUDA_FAST_MATH=1 -D CUDA_ARCH_BIN=7.0 -D WITH_CUBLAS=1 -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules -D HAVE_opencv_python3=ON -D PYTHON_EXECUTABLE=~/.virtualenvs/opencv_cuda/bin/python -D BUILD_EXAMPLES=ON ..
For one more point, check the install path in Python 3 section of CMake output. We will use install path it. So please leave a note of install path.
Step 6: Compile OpenCV and Create a Symbolic link
If cmake exited with no errors, you then compile OpenCV with the following command.
And a word of caution, before you run the next commands, it will take a minimum of 45mins to a maximum of 5hrs, make sure you don’t get time out the ssh connection if you are accessing it via putty, keep pressing Enter every 15 mins.
make -j$(nproc)
sudo make install
sudo ldconfig
ls -l /usr/local/lib/python3.6/site-packages/cv2/python-3.6cd ~/.virtualenvs/opencv_cuda/lib/python3.6/site-packages/
##Next, create a sym-link to your virtual environment: ##ln -s /usr/local/lib/python3.6/site-packages/cv2/python-3.6/cv2.cpython-36m-x86_64-linux-gnu.so cv2.so
Then, we are going to create a sym-link the OpenCV Python bindings into your Python virtual environment. As mentioned in the previous step, we know that the install path is /usr/local/lib/python3.6/site-packages/cv2/python-3.6.
To confirm, you can use the ls command:
You can see the name of my OpenCV Python bindings is cv2.cpython-36m-x86_64-linux-gnu.so (you may have a similar name of your own built bindings).
Next, create a symlink to your virtual environment:
Remember to take a second to check your file paths because ln will silently fail if the path of OpenCV bindings is not correct.
Verify the Installation — OpenCV Python
Running OpenCV + CUDA enabled codes instead of OpenCV codes and this time it seems to be working.
So most of the OpenCV functions can be wrapped with CUDA, to see the available functions, run the command in the Python IDE.
## To check the supported methods for opencv in CUDA ##
import cv2print(dir(cv2.cuda))
Check it’s corresponding outputs in WinSCP connected with my instance.
Outputs from OpenCV CUDA enabled codes, where I resized and converted color channel to gray.
So when you log in to your instance next time and want to enter into the OpenCV CUDA environment, use the command.
Here the name of my virtual environment is “opencv_cuda”.
source ~/.virtualenvs/opencv_cuda/bin/activate
A big congratulations, if you have reached this point both in the article and also while implementation, now you can deploy production-grade applications with OpenCV CUDA support.
References
It would have not been possible for me, without these amazing references I found on the internet.
To know more in detail, you can visit any of them.
https://github.com/spmallick/learnopencv/tree/master/Getting-Started-OpenCV-CUDA-Module
https://gist.github.com/raulqf/f42c718a658cddc16f9df07ecc627be7
https://medium.com/@Linh.NG/installing-opencv-3-3-0-on-ubuntu-16-04-lts-7db376f93961
History of Commands :
As I said, I am adding the consolidated codes, I ran so far and you can follow them along with the corresponding screenshots in the above.
Finally, as a word of precaution, it costs me $48 USD for making this post and more miscellaneous costs for my previous instance breakage, as I had to terminate multiple instances before due to improper bindings.
Make sure you use the corresponding GPU instances based on your usage.
Until then, see you next time.
Article By:
BALAKRISHNAKUMAR V
