I’m working on a new personal project that involves recognizing and tracking vehicles in a video feed from a camera overlooking an all-way stop intersection. An early prototype of the system running on my MacBook Pro (more in a later blog post) promised technological feasibility but using Convolutional Neural Networks (CNN) for object detection and tracking in each frame turned out too processing intensive for real time.
The hardware I’m using is an old (Sandy Bridge) i7 desktop computer that was fitted with 32GB of DDR3 RAM and an NVIDIA GeForce GTX1070 graphics card. The goal is to use the GPU for Neural Network Intensive learning tasks.
Ubuntu 18.04 (Bionic Beaver)
For the installation of Ubuntu 18.04 I’ve added a blank new SSD drive to my desktop machine. I’ve used
Etcher (https://etcher.io) to create a bootable USB of the latest Ubuntu ISO (http://releases.ubuntu.com/18.04/). Booted from the USB, selected
Install Ubuntu, followed through the install steps guide. During hard drive partitioning, I selected
Install NVIDIA Drivers
Once booted into the fresh Ubuntu install, the first thing I did was to install the NVIDIA drivers from the ‘official’ NVIDIA package repository. At the time of writing this was version
sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt update
sudo apt install nvidia-396
After rebooting, running the
nvidia-smi command should print out details about the graphics card and driver. I also installed the settings pane.
sudo apt install nvidia-settings
After the NVIDIA driver installation, I proceeded with installing a number of graphics and video libraries that are required for building
sudo apt install git
sudo apt install freeglut3 freeglut3-dev libxi-dev libxmu-dev
sudo apt install build-essential cmake unzip pkg-config
sudo apt install libjpeg-dev libpng-dev libtiff-dev
sudo apt install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
sudo apt install libxvidcore-dev libx264-dev
sudo apt install libgtk-3-dev
sudo apt install libatlas-base-dev gfortran
sudo apt install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev
The NVIDIA CUDA Toolkit provides a development environment for creating high performance GPU-accelerated applications. With the CUDA Toolkit, you can develop, optimize and deploy your applications on GPU-accelerated embedded systems, desktop workstations, enterprise data centers, cloud-based platforms and HPC supercomputers. The toolkit includes GPU-accelerated libraries, debugging and optimization tools, a C/C++ compiler and a runtime library to deploy your application. GPU-accelerated CUDA libraries enable drop-in acceleration across multiple domains such as linear algebra, image and video processing, deep learning and graph analytics.
First, we install
CUDA as this is a compile time dependency for
OpenCV. Download the
runfile installer from https://developer.nvidia.com/deep-learning
It will complain about an “incompatible setup”, which we can safely ignore. Also, we need to say ‘NO’ to driver, and yes to everything else.
/opt/cuda/9.2 as install dir and say yes to symlink to
/usr/local/cuda. The following instructions assume that installation directory.
sudo sh cuda_9.2.148_396.37_linux.run
sudo sh cuda_220.127.116.11_linux.run
CUDA 9.2 and the patch update, we can add the
CUDA paths to our profile.
and also update
ldconfig like so:
echo "/usr/local/cuda/lib64" >> /etc/ld.so.conf.d/cuda.conf
The NVIDIA CUDA Deep Neural Network library (cuDNN) is a GPU-accelerated library of primitives for deep neural networks. cuDNN provides highly tuned implementations for standard routines such as forward and backward convolution, pooling, normalization, and activation layers. cuDNN is part of the NVIDIA Deep Learning SDK.
CUDA, we can download
cuDNN from https://developer.nvidia.com/cudnn. Installation is a straightforward unpacking fo the archive, and copying files to our
CUDA installation directory.
tar xjf cudnn-9.2-linux-x64-v18.104.22.168.tgz
sudo cp cudnn-9.2-linux-x64-v22.214.171.124/cuda/lib64/libcudnn* /opt/cuda/9.2/lib64
sudo cp cudnn-9.2-linux-x64-v126.96.36.199/cuda/include/cudnn.h /opt/cuda/9.2/include/
We first install the dev libraries for
Python 3, as well as the
sudo apt install python3-dev
sudo python3 get-pip.py
sudo rm -rf get-pip.py
Next we setup
virtualenv to work in Python virtual environments, which is best practice for Python development.
sudo pip install virtualenv virtualenvwrapper
# virtualenv and virtualenvwrapper
Finally, we create a virtual environment for our
OpenCV 4 development:
mkvirtualenv py3cv4 -p python3
At the time of writing,
OpenCV 4 is still in development, with a release date in fall. I wanted to take advantage of the many improvements added to the deep learning (
dnn) module in the upcoming release. It’s straightforward to compile
OpenCV 4 from the GitHub sources.
Clone Git Sources
First we clone
cd ~ && mkdir -p Repositories/github.com/opencv
git clone https://github.com/opencv/opencv.git
git clone https://github.com/opencv/opencv_contrib.git
Configure and Build
The following steps will configure the
OpenCV 4 build. The instructions assume the paths for the git repositories and
CUDA libraries are like described above.
cd opencv && mkdir build
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D INSTALL_PYTHON_EXAMPLES=ON -D INSTALL_C_EXAMPLES=OFF -D OPENCV_EXTRA_MODULES_PATH=/home/nicbet/Repositories/github.com/opencv/opencv_contrib/modules -D PYTHON_EXECUTABLE=~/.virtualenvs/py3cv4/bin/python -D BUILD_EXAMPLES=ON -D WITH_CUDA=ON -D CUDA_TOOLKIT_ROOT_DIR=/opt/cuda/9.2 ..
Compilation of the code can take a little while - about 35 minutes on my desktop machine.
make install I initially hit an error as we were building from
./build directory. We need to kick off header generation like so, then
make install will complete. It’s an open bug tracked here: https://github.com/opencv/opencv/issues/10771
python ../modules/python/src2/gen2.py ../build/modules/python_bindings_generator ../build/modules/python_bindings_generator/headers.txt
sudo make install
Link to Virtualenv
Finally, we will link the
OpenCV module to our Python virtual environment.
ln -s /usr/local/lib/python3.6/site-packages/cv2.cpython-36m-x86_64-linux-gnu.so cv2.so
Test OpenCV 4 Installation
We can test our installation by invoking a Python REPL:
>>> import cv2
Tensorflow with GPU acceleration, I followed the instructions from the official website. Even though we installed
CUDA 9.2 earlier, the
Tensorflow packages require at the time of writing
CUDA 9.0. That should change with the upcoming version of the
CUDA libraries: next
Tensorflow will target
CUDA 9.2 (they are skipping 9.1), and
CUDA 10 will be fully backward compatible to
CUDA 9.2, so we will finally have a way out of the dependency chicken game.
sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub
sudo dpkg -i cuda-repo-ubuntu1604_9.1.85-1_amd64.deb
sudo dpkg -i nvidia-machine-learning-repo-ubuntu1604_1.0.0-1_amd64.deb
sudo apt-get update
sudo apt-get install cuda9.0 cuda-cublas-9-0 cuda-cufft-9-0 cuda-curand-9-0 \
cuda-cusolver-9-0 cuda-cusparse-9-0 libcudnn7=188.8.131.52-1+cuda9.0 \
sudo apt-get update
sudo apt-get install libnvinfer4=4.1.2-1+cuda9.0
Later when Tensorflow 1.9 has CUDA 9.2 Support
See this Github issue for a discussion around
CUDA 9.2. I’ve decided to take the easy way out and just have
CUDA 9.0 and
CUDA 9.2 installed in parallel, with
OpenCV 4 using
CUDA 9.2 and
CUDA 9.0 libs.
sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1710/x86_64/7fa2af80.pub
echo "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1710/x86_64 /" | sudo tee /etc/apt/sources.list.d/cuda.list
sudo apt update
sudo apt -o Dpkg::Options::="--force-overwrite" install cuda-9-2 cuda-drivers
Test Tensorflow GPU
Again, we can test the install by invoking a Python REPL:
>> from tensorflow.python.client import device_lib
And there you have it. The next few of blog posts will describe the vehicle recognition and tracking system in more detail.