Why Deep Learning over Traditional Machine Learning?

Artificial Intelligence is on a rage! All of a sudden every one, whether understands or not, is talking about it. Understanding the latest advancements in artificial intelligence can seem overwhelming, but it really boils down to two very popular concepts Machine Learning and Deep Learning. But lately, Deep Learning is gaining much popularity due to it’s supremacy in terms of accuracy when trained with huge amount of data.

Just to show you the kind of attention Deep Learning is getting, here is the Google trend for the keyword:

Trend of “Deep Learning” in google

The software industry now-a-days moving towards machine intelligence. Machine Learning has become necessary in every sector as a way of making machines intelligent. In a simpler way, Machine Learning is set of algorithms that parse data, learn from them, and then apply what they’ve learned to make intelligent decisions.

Examples of Machine Learning are everywhere. It’s how Netflix knows which show you’ll want to watch next or how Facebook recognises your friend’s face in a digital photo. Or how a customer service representative will know if you’ll be satisfied with their support before you even take a CSAT survey.

The thing about traditional Machine Learning algorithms is that as complex as they may seem, they’re still machine like. They need lot of domain expertise, human intervention only capable of what they’re designed for; nothing more, nothing less. For AI designers and the rest of the world, that’s where deep learning holds a bit more promise.

What is Deep Learning?

Practically, Deep Learning is a subset of Machine Learning that achieves great power and flexibility by learning to represent the world as nested hierarchy of concepts, with each concept defined in relation to simpler concepts, and more abstract representations computed in terms of less abstract ones.

Elaborately, a deep learning technique learn categories incrementally through it’s hidden layer architecture, defining low-level categories like letters first then little higher level categories like words and then higher level categories like sentences. In the example of image recognition it means identifying light/dark areas before categorizing lines and then shapes to allow face recognition. Each neuron or node in the network represents one aspect of the whole and together they provide a full representation of the image. Each node or hidden layer is given a weight that represents the strength of its relationship with the output and as the model develops the weights are adjusted.

Deep Learning Architecture

Distinctive Features Of Deep Learning

A big advantage with deep learning, and a key part in understanding why it’s becoming popular, is that it’s powered by massive amounts of data. The “Big Data Era” of technology will provide huge amounts of opportunities for new innovations in deep learning. As per Andrew Ng, the chief scientist of China’s major search engine Baidu and one of the leaders of the Google Brain Project, “The analogy to deep learning is that the rocket engine is the deep learning models and the fuel is the huge amounts of data we can feed to these algorithms.”

Deep Learning requires high-end machines contrary to traditional Machine Learning algorithms. GPU has become a integral part now to execute any Deep Learning algorithm.

In traditional Machine learning techniques, most of the applied features need to be identified by an domain expert in order to reduce the complexity of the data and make patterns more visible to learning algorithms to work. The biggest advantage Deep Learning algorithms as discussed before are that they try to learn high-level features from data in an incremental manner. This eliminates the need of domain expertise and hard core feature extraction.

Another major difference between Deep Learning and Machine Learning technique is the problem solving approach. Deep Learning techniques tend to solve the problem end to end, where as Machine learning techniques need the problem statements to break down to different parts to be solved first and then their results to be combine at final stage. For example for a multiple object detection problem, Deep Learning techniques like Yolo net take the image as input and provide the location and name of objects at output. But in usual Machine Learning algorithms like SVM, a bounding box object detection algorithm is required first to identify all possible objects to have the HOG as input to the learning algorithm in order to recognize relevant objects.

Usually, a Deep Learning algorithm takes a long time to train due to large number of parameters. Popular ResNet algorithm takes about two weeks to train completely from scratch. Where as, traditional Machine Learning algorithms take few seconds to few hours to train. The scenario is completely reverse in testing phase. At test time, Deep Learning algorithm takes much less time to run. Whereas, if you compare it with k-nearest neighbors (a type of machine learning algorithm), test time increases on increasing the size of data. Although this is not applicable on all machine learning algorithms, as some of them have small testing times too.

Interpretability is the main issue why many sectors using other Machine Learning techniques over Deep Learning. Let’s take an example. Suppose we use deep learning to calculate the relevance score of a document. The performance it gives is quite excellent and is near human performance. But there’s is an issue. It does not reveal why it has given that score. Indeed mathematically you can find out which nodes of a deep neural network were activated, but we don’t know what there neurons were supposed to model and what these layers of neurons were doing collectively. So we fail to interpret the results. Which is not in case of Machine Learning algorithms like decision trees, logistic regression etc.

When to use Deep Learning or not over others?

1. Deep Learning out perform other techniques if the data size is large. But with small data size, traditional Machine Learning algorithms are preferable.
2. Deep Learning techniques need to have high end infrastructure to train in reasonable time.
3. When there is lack of domain understanding for feature introspection, Deep Learning techniques outshines others as you have to worry less about feature engineering.
4. Deep Learning really shines when it comes to complex problems such as image classification, natural language processing, and speech recognition.

References-

When Does Deep Learning Work Better Than SVMs or Random Forests?

When to Choose Deep Learning

Machine Learning vs. Deep Learning