Lane Change Detection — Computer Vision at Next stage
Leading innovations in OpenCV and Image Processing
Hey Guys, get excited because in this tutorial we will look at how to use Computer Vision and Image Processing to detect if a car is changing lanes while the cars are on the road, so without further delay let’s start -
You must have heard that using OpenCV haar cascade files detection of faces, eyes, or objects like car, bus, etc. can be done but what then? So let’s use this simple detection method to build something cool.
1. Dataset
Video files of cars on a road are used as a dataset in this tutorial. Also, we can detect cars in an image using image datasets but here as the car is changing lanes we give an alert with a pop-up window, so for these dynamics, video input is more feasible.
2. Input
The first step is to give the inputs to be used in the tutorial — a haar cascade file of OpenCV to detect the coordinates of a car, a video file of cars on a road -
cv2.VideoCapture() method is used for capturing the input video, a video is generally 25 images/frames per second (fps). After capturing the input, frames are extracted using a loop, and using the coordinates detected by the haar cascade file of a car, we draw a rectangle around the car in a loop to get consistency while performing other operations on the captured frame.
In OpenCV BGR is used and not RGB, so (0,0,255) will draw a red rectangle over the car and not blue.
3. Image processing
We use the frame but if the frame is in very high resolution it will slow down the operations performed and in addition, the frame consists of noise which can be reduced using a blur, here Gaussian blur is used.
Now, let’s see some image processing concepts -
3.1 HSV frame
In this, we use an HSV frame obtained from the frame captured by cv2.VideoCapture() to highlight only the points at which cars are making a turn and blackout the remaining road & cars moving straight on the road. Upper and Lower Thresholds are set to define the range of color in HSV to see the points at which the car is changing lanes and to be used as a mask for the frame. Below is the snippet of code used to obtain this -
3.2 Erosion & Dilation
Erosion and Dilation are two fundamental morphological operations used in image processing. The erosion operator has an effect of local minimum over the area of the kernel, the kernel is a template or mask. Erosion is used to reduce the speckle noise in the image, speckles are eroded from the object boundaries in an image. Dilation is the convolution of an image and a kernel and it has the effect of a local maximum operator. Dilation is applied to regain some lost area as pixels are added to smoothen the boundaries of objects in an image.
The mask generated from the first step in the HSV frame is now treated with the fundamental morphological operations (erosion & dilation). The resulting frame is generated by applying Bitwise AND operation between the frame and the mask to the get ROI (region of interest).
3.3 Lane Detection
The canny edge detector is used with Hough Line Transform for the detection of lanes.
4. Contours
Algorithms such as canny edge detector are used to find edge pixels that separate the edges in an image but it doesn’t tell how to find objects or entities as we can’t combine/assemble some points and edges, here we can use the concept of contours implemented in OpenCV as cv2.findContours().
Definition — “A contour is a list of points that represent a curve in an image.” Contours are represented by sequences (Sequence is a linked list of structures), every sequence encodes the information about the location of the next point. We run cv2.findContours() more than one time in ROIs to get the entities, then cv2.drawContours() is used to draw contours area. Contour can be a point, edge, polygon, etc. so while drawing contours we do polygon approximation, finding the length of an edge and area of a region. The function cv2.drawContours() works by drawing a tree (data structure) starting with the root node then joining subsequent points, bounding boxes, and freeman chain codes are used in the process.
Another important task after finding a contour is to match them. Matching contours means we have two separate computed contours to compare with each other or a contour to compare with an abstract template.
5. Moments
We can compare two contours by computing contour moments. “A moment is a gross characteristic of the contour computed by adding all the pixels of the contour.”
Types of moments -
Spatial moments: m00, m10, m01, m20, m11, m02, m30, m21, m12, m03.
Central moments: mu20, mu11, mu02, mu30, mu21, mu12, mu03.
Hu moments: There are seven Hu moments (h0 — h6) or (h1 — h7), both notations are used.
We calculate the moments and fit ellipse on the points using cv2.fitEllipse(). The angle is found from the contour & the moments, as 45-degree rotation is required to change lanes, which is considered as a threshold for the car turning angle.
Now instead of just printing the detection of changing lanes, we can use Tkinter for a simple pop-up window to alert the change.
Measuring angle with Greenline with the rectangle drawn on the car in the frame
6. Summary and Future Scope
In this tutorial, a small demonstration of smart car navigation is explored using the lane change detection method.
Computer Vision is rapidly growing and its applications are advancing not just in local navigation of automobiles but navigation on Mars and in fields of product inspection, even medical applications are being developed and used to detect cancer and tumors in the X-ray images in the early stages.
Hurray, you made it to the end. Awesome. Click here to get the source code at my GitHub account.
Feel free to ask about any doubts because we are learning the same as the machines Hitesh Valecha…
References —
- Bradski, Gary and Kaehler, Adrian, Learning OpenCV: Computer Vision in C++ with the OpenCV Library, O’Reilly Media, Inc., 2nd edition, 2013, @10.5555/2523356, ISBN — 1449314651.
- Laganiere, Robert, OpenCV Computer Vision Application Programming Cookbook, Packt Publishing, 2nd edition, 2014, @10.5555/2692691, ISBN — 1782161481.
Well, as innovation never stops, I invite you to read another innovative blog on Ear Biometrics.
