Seeing the Road — Getting Started with Self-Driving Cars
In a strange sadistic episode I decided to jump headfirst into self-driving cars and AI as a side-project. Having a consulting business, a toddler and virtually no free time I thought, “take some courses on the side? Now that’s a great idea!”[1] . At the least maybe I can de-mystify some of the technology being it
In taking the self-driving course, I frequently talk to people about the technology. What I found in talking with people about self-driving cars is that most people are either surprised at the current level of technology or horrified by it. Which is a natural reaction. Driving has become such a distinctly human endeavor that it is impossible for most people to grasp that a computer can essentially perform this task to a much higher degree with little to no driver error. That, or they believe self-driving cars are so far off that they don’t have to worry about it.
But underestimating what computers can do is not unique to driving. As a shamelessly bad chess player in my younger days I idolized the passion and intensity of the grandmasters. Using terms like, creative, aggressive, unrelenting, Kasparov was really one of the last grandmasters before falling in a series of televised matches in 1997. It became the tipping point for computer algorithms in chess and they haven’t looked back. Nowadays, even a moderate $20 game are rated higher than grandmasters. The era of chess being the pinnacle of human ability fell rapidly and has not looked back.
In the same way, self-driving cars will surpass human abilities in a rapidly shorter time frame. Which is good because I can’t imagine the stress of teaching my daughter to drive.
Computer Vision
A human driver doesn’t have a lot of sensors to work with but they do have an excellent visual system. And driving is tailored to work with it. Traffic signs, lane markings, and turn signals (even on BMWs) provide cues for navigating through the environment. But seeing something is not the same as understanding its context. We can see the road markings but understanding their meaning is completely different. The perception portion of driving is complex for a computer (most drivers too, seriously watch the road) and most believe this is where humans excel. If a self-driving car is to work on the road they will be required to not only see the road but understand the context behind those markings.
Much of the technology in computer vision for self-driving cars now reflects what we know (or guess) about human vision. The eye performs a series of image transformations in the cones extracting colors, lines and patterns and forwards that information through to the brain to process. It never captures a whole image but simply focuses different areas to look for patterns. That is why you can count the ball movements but never see the gorilla.
NVIDIA and others have taken this concept and integrated it within a learning algorithm, called a Convolution Neural Network(CNN). The CNN is an important tool in machine vision and has been proven to be effective in a huge variety of computer vision fields. In driving we want to focus on the visual indicators on the road and the CNN can be taught to identify road features simply by training it with videos of driving.
NVIDIAs implementation even went so far as to teach a car to navigate a road solely using camera vision. And if you are under the assumption that this type of technology is limited to big universities or companies with huge research budgets, here is the same technique applied to a simulated course as a small part of the 3-month self-driving course;
The computer vision field as it applies to self-driving cars is very far along. Even the concepts here are rudimentary in comparison to the work done by Google, BMW, Mercedes and virtually every other car manufacturer with respect to computer vision.
Maybe we need a Man v Machine car challenge….
[1] If you are familiar with them, I am working through a couple Nanodegree programs from Udacity specializing in AI and Self-Driving cars. They are great introductions to the field and engage some top industry personnel. They cover a huge amount of information and at times it feels like drinking from a firehose.
