A Critical Appraisal of Deep Learning
The field has seen some strong progress, but the more we know, the more we realize we know nothing.
Introduction
Everyone and their grandparents are talking about it: Artificial Intelligence, Deep Learning (DL), Machine Learning, Robotics, etc… Sometimes all those terms at once within the same sentence, sometimes as synonyms. If there’s one sure thing, these subjects have gained in popularity, to the point the general public has been having growing expectations. Major progress in facial recognition, image classifiers, in AI-vs.-human gaming (GO, DOTA), or self-driving cars, have created understandable hype. The hype itself was also driven by the over-zealous promises from CEOs and leading researchers.
But hype only survives when concrete applications come into place, and this is still lagging. As time goes by, a slow disenchantment is starting to kick-in in some areas. Chatbots have not lived up to their hype. Self-driving cars and Neural Network as a whole could be next. In this article, I will review Gary Marcus’ critical appraisal of Deep Learning¹, and complement it with personal commentary and some resources to go further.
Deep Learning Systems’ Challenges
In his paper (2018), Marcus discusses deep learning open challenges.
Deep learning is data-hungry
As an example, Humans need a few trials or data examples to transform those into actionable.
- Marcus takes the example of a made-up word, a ‘schmister’. In our example, it’s defined as ‘having a sister over the age of 10 but below the age of 21 years’. With one example, you can easily tell if you have a schmister, or if friends do. And more, by deduction, you can deduce your parents probably don’t have any.
Marcus here underlines deep learning’s current lack of a mechanism for learning abstractions through explicit, verbal definition. While instead, it works best when there are thousands, millions or even billions of training examples, as in DeepMind’s performance on games.
In short, in problems where data are scarce, deep learning is not yet an ideal solution. Yet…
