The world’s leading publication for data science, AI, and ML professionals.

Four Mistakes to Avoid While Learning AI

AI is wonderful, but we should approach it the right way.

Getting Started, ARTIFICIAL INTELLIGENCE

Photo by Markéta Marcellová on Unsplash
Photo by Markéta Marcellová on Unsplash

Learning artificial intelligence saved me from working at a boring job.

I started studying AI right away after finishing my undergrad in aerospace engineering. After 4 months of studying courses, papers, books, and trying things out for myself, I landed a dream job: A R&D project to build a real-time bidirectional sign-language translator. Something that didn’t exist. I was about to create the future!

Those 4 months allowed me to escape a life I would surely have hated. I wanted something more exciting, like reverse-engineer our brain and mind. I deep-dived into a new, unexplored world that showed me its lights and shadows. Here are 4 mistakes I learned to avoid throughout my journey into the AI field. Enjoy!


Starting with the theory

"Don’t start with definitions and theory. Instead, start by connecting the subject with the results you want."

If you’ve read some of my articles, you may have noticed I love theoretical AI. It has a philosophical touch to it that amazes me. I like to ponder about the future of the field and what it’ll mean for us. How will AGI shape our lives? Will we create it without us even noticing? Will it be dangerous or peaceful?

I also like to read about the historical foundations. How were the cognitive sciences and computer science separated at birth in the mid-20th century? How did symbolic AI fall from grace to let neural networks step in, reaching eventually the hegemonic status they enjoy today? Why has AI suffered from winter periods?

And I also like to know where things come from. What is a convolution? How did transformers, based solely on attention, overthrow other paradigms in the last few years? How is it possible that adding parameters makes models qualitatively more powerful?

All these questions attract me like a moth to a light bulb. Yet, when I started learning AI and took Andrew Ng’s and Geoff Hinton’s courses back in 2017, I knew the philosophical, historical, and theoretical sides of AI wouldn’t get me a job. I wanted to work on a real-life project. I wanted to get my hands dirty. I wanted to get the know-how knowledge of actually building something from scratch. Books couldn’t teach me that. Stories and deep reflections couldn’t teach me that. Practical knowledge and useful skills are what I needed to keep the project from falling apart.

But the path isn’t equal for everyone. There are two situations in which starting with theory would make sense. First, if you’re studying computer science at the university you’ll probably learn the AI’s bases even if you don’t want to. But because you’re still at university – and the potential success of a project isn’t resting on your shoulders – it’s probably worth it to take advantage of it. Theory and history aren’t the most urgent aspects of AI, but they’re for sure an edge when competing with other people.

The second situation is if you’re headed to academia. Scholars and researchers know the pillars of the field they study. They don’t simply use AI to build a company or a project; they create those algorithms. They are the ones who know why transformers are getting more success than other models. They know why symbolic AI didn’t work. They know because they created it. And they also created deep learning. Every great breakthrough in AI came from scholars that knew the theory behind the practice.


Dismissing the theory entirely

"What we need most of all in AI are thinkers, not programmers."

  • Sridhar Mahadevan, computer scientist

Starting with the theory isn’t a good idea if you want to get into the AI industry. But forgetting that AI has a history and a theoretical background sustaining practical applications isn’t smart. You may not invent the next great deep learning algorithm, but knowing the reasons behind ReLu’s popularity or why every neural network used batch normalization could help you make better decisions.

The main problem with building models without knowing the "whys" is that if circumstances deviate you from the usual path, you may not have the knowledge to get back on track. If your model starts getting crazy results, you may need to debug it. Knowing what causes them is the only way to find the ghost in the machine.

The more practical knowledge—coding, neural net architectures, state-of-the-art (SOTA) models, and even statistics and algebra – is of no use if you don’t understand the problem you’re trying to solve. Citing computer scientist Sridhar Mahadevan, "What we need most of all in AI are thinkers, not programmers."

By now, you’ve probably realized that finding a compromise solution between practical and theoretical knowledge is key to walk the right path: don’t start on theory, but don’t forget it either. Maybe the best option is what Jason Brownlee calls a top-down approach: A natural way to learn skills such as reading, driving, or coding. Many people learn to code in their spare time with side projects. Get excited with AI, make it spark your curiosity, make it a quest of discovery. Then, you will find the theory interesting.


Starting from deep learning

"The future depends on some graduate student who is deeply suspicious of everything I have said."

  • Geoffrey Hinton, ‘Godfather’ of deep learning

Deep learning is undoubtedly the most successful AI paradigm. Symbolic AI reigned for 3 decades but it didn’t achieve an ounce of what deep learning has achieved. Since 2012, deep learning has grown so much that it casts shadows over its parent fields. DL = ML = AI isn’t correct, but it’s how the media depicts the field to the general population; deep learning has become a synonym of AI.

Deep learning is the fun, it’s the success, it’s the money. Yet, starting a house from the roof won’t let you build the walls. There’s a reason why Andrew Ng’s most popular course starts with linear regression and linear algebra. Because it gives you an invaluable toolkit to frame deep learning within a picture that includes other techniques and methods that came before it and paved its foundations.

If you start with deep learning, you won’t find a reason to go back and learn other machine learning algorithms, learn algebra or statistics, or other techniques that were successful in the past – and are still today. You’ll have the powerful tool; you’ll have the hammer. But for a hammer, everything looks like a nail. Deep learning has found the most fame and popularity, but there are problems more efficiently solved with other methods. Deep learning requires huge datasets and tons of computing power. Not every company can afford those resources.

If after careful consideration you find that starting with deep learning is the best way to your goals, then go ahead. But don’t follow the trends blindly, or you may hit a wall you won’t be able to climb. Deep learning is the definition of success in AI now, but it may not be forever. If another paradigm overthrows it, you better have jumped from the bandwagon on time. Having said this, you’ll have to save deep learning a spot in your toolkit eventually. That’s beyond question.


Getting too hyped about AI’s state-of-the-art

"Machine intelligence is the last invention that humanity will ever need to make."

  • Nick Bostrom

This one happened to me. Not because I got confused by popular culture’s depiction of AI with films like The Matrix or I, Robot, but because even prestigious media outlets weren’t telling the real story. The same happened last year with GPT-3 (as I covered in this long-form article). The hype got wild because people were portraying the system as all-powerful when the reality was more nuanced.

AI works, and its applications affect – or better, impact – a wide array of industries. AI can provide important competitive advantages. AI attracts a lot of funding and interest and moves huge amounts of money – and business people do anything to keep the money. It’s not a secret that not many companies are truly using AI or deep learning. They talk about AI-powered products because it sells, but their solutions could be perfectly attained with simpler, less deluxe technologies.

If you want to know the state of the art, you need to read the papers, to listen to the experts that don’t depend on the success of the field. You won’t hear the same opinions from computer scientists as from philosophers or AI ethicists. Find the right sources and you’ll be able to compare money’s story with what’s actually being done.

We’re still far from AGI. We’re advancing at a considerable pace, but we’re still far. Larger models are the trend, but we don’t know where it’s leading us. We can make assumptions but no one really knows. Anyway, it’s normal to lose a bit of hype when you enter a new industry. Things shine more from the outside. This happens in every industry, although not always to the same degree. AI is particularly overhyped. Taking this fact into account is what matters.


Conclusion

AI is a wonderful field to study or work in. Each year we’re seeing new things coming out and it’s getting more and more ingrained in our lives. The future will be AI-driven, so we better keep our eyes open. But because AI receives so much attention, it’s important to be especially careful. Here are the four mistakes summarized in one sentence each:

  • Follow a top-down approach.
  • Find a balance between practical and theoretical AI.
  • Don’t let trends dictate your path.
  • Don’t let the hype convert AI into a mirage.

Disclaimer: The Advice in this article comes from my own experience. Other people may have had different experiences and may disagree with me. Think about this article through those lenses: It’s an opinion that I hope can help you somehow!


Subscribe for more content on AI, philosophy, and the cognitive sciences!

Recommended reading

5 Reasons Why I Left the AI Industry

GPT-3 – A Complete Overview

GPT-3 Scared You? Meet Wu Dao 2.0: A Monster of 1.75 Trillion Parameters


Related Articles