The world’s leading publication for data science, AI, and ML professionals.

The Hard Truth: Data Science Isn’t for Everyone

Hard work and effort sometimes aren't enough.

DATA SCIENCE | OPINION

Photo by Thought Catalog on Unsplash
Photo by Thought Catalog on Unsplash

We like to believe hard work trumps everything. It’s false.

Whatever we want to achieve professionally, several factors always affect the outcomes. We love the idea of "hard work = success" because effort is a controllable factor that recognizes our merit. However, elements outside our control also influence our lives. I could train hard to play at the NBA, but at 5’11 it doesn’t matter how much I try, the odds are strongly against me.

Hard work can compensate for the lack of other factors, but only to an extent. Believing anyone could be anything just by changing the direction and magnitude of their efforts is highly misleading. We’re bound by our situation and circumstances. They define us. There’s a famous phrase by Spanish philosopher José Ortega y Gasset that perfectly captures this idea:

"I am I and my circumstance."

In this article, I’ll explain why Data Science – or virtually any other discipline – isn’t for everyone. I’ll describe three forces that influence how prepared we are to work in this field.

Disclaimer: This is an opinion article. Feel free to open the discussion in the comments!


Lack of intelligence – Missing potential

AI experts often use the concept of artificial general intelligence (AGI) to refer to systems that have human-level intelligence. It’s a strange concept because we put all humans into the same bag. However intelligent you are, you have human-level intelligence.

Yet, when we zoom in, we realize intelligence varies highly across people. Einstein, which often serves as an example of human brilliance, was vastly more intelligent than I am. And the same could be said to the other end of the spectrum. Humans are all approximately equally intelligent, but only looking from afar.

I’m not referring to intelligence in the IQ sense of the word. Although not scientifically robust, I highly value Howard Gardner’s hypothesis of multiple intelligences. When I say the word "intelligence" I understand the concept as a multidimensional set of skills. I can be good at language and horrible at dancing. Another person can have fantastic people skills and lack introspection. Math and logic make up just one dimension. And it is the people who excel at this dimension who are better suited to learn and work in technical, math-based disciplines, such as data Science.

Technical people share a specific set of features: They’re logical and analytical, problem-solvers, have critical-thinking skills, and their abstract understanding is top-notch. To some degree, they were born with the potential to be this way. We’re not a tabula rasa when we’re born; we come to this world slightly pre-programmed with some latent aptitudes. These people happen to belong to the group that’s potentially adequate for these fields.

On the other hand, people whose biological pre-program hasn’t paid much attention to math and logic will have a harder time in data science. Even if they’d love to become a data scientist and they put huge amounts of effort. The reason is they’re lacking a key piece of the puzzle; the adequate potential is missing. It isn’t impossible – environmental forces may compensate a lower degree of innate abilities – but the odds aren’t in their favor.


Lack of education – Unfulfilled potential

If intelligence biologically limits our capabilities, education limits them culturally. As an analogy, intelligence would be the size of our "capability bucket" whereas education would be the degree to which it’s full or empty. Interestingly, the capability bucket is often bigger than we assume and it’s an appropriate education that’s missing.

Our potential is in part innate but the degree to which it’s fulfilled is learned. If both my parents were engineers – other things being equal—I’d have had an easier time learning math and physics at school than other students. If I take a data science course in which the professor is deeply passionate about the topic, I’d learn faster and better than those who take a different course. Environment explains – perhaps even more than innate ability – the degree of mastery we can achieve in a given area.

And other circumstances can affect the quality of our education: Socioeconomic status, historical events, the closeness of familiar relationships, lack of opportunities, or something so simple as the place where we were born. Even if my parents were engineers – again, everything else being equal – I’d be better off living in New York than living in Detroit, Mumbai, or Nairobi. The available resources or the conditions of infrastructure also affect the quality of our education. People who can’t access good educational services are at an unfair disadvantage.

In the end, dormant intelligence is as useful as no intelligence. It doesn’t matter that the potential is there if it stays unfulfilled. We have to receive education and support to grow and develop our innate skills. I was prepared to work at AI because my father had studied physics and because I studied aerospace engineering and then enrolled in several high-level AI courses. Otherwise, I would have remained unaware of what I could do and would never have tried to work at AI.


Lack of knowledge – Unused potential

The field of data science has become incredibly attractive because people have the illusion that it’s an easy-to-enter and high-reward field. Harvard Business Review called it the "sexiest job of the 21st century." Dozens of online courses promise that only a few months of hard study can prepare anyone to land a data scientist position at a decent company. My personal experience further reinforces this notion: I started studying AI in September 2017 and got a job at an AI startup in January 2018. Four months of study seems very little time to learn enough of a field to land a job.

What solves this apparent incoherence is that there are different approaches to learn anything. If I had started studying AI theory since Turing and the early ideas of symbolic AI, I’d still be learning expert systems 4 years later. Instead, I took a brief course to get some basic knowledge on machine learning and then went directly to learn deep learning and coding. I used a top-down approach: Instead of setting the bases firmly, I learned by playing with the most immediate applications I could find.

However, this approach isn’t risk-free. If a person without a good math/statistical background – my example isn’t valid because I was an aerospace engineering undergrad – thinks that learning Python, TensorFlow, and some key architectures and models is enough to work in AI, they’d be strongly mistaken.

Recently, some articles about this topic – expressing diverse opinions – have been published in Towards Data Science. Soner Yıldırım wrote a popular post last month stating that a data scientist should know how to do the tasks of a data engineer. Earlier this week, Chris The Data Guy wrote a controversial article stating that an ML engineer doesn’t need math. And two days ago, Sarem Seitz published a piece arguing the exact opposite.

There’s always a trade-off. If we choose to spend more time coding and doing real projects, our theoretical understanding would be more shallow. If we choose to learn the groundwork and get in touch with every theory, algorithm, and technique, our practical skills would be more shallow. Whatever you do, if you don’t dedicate enough time to learn about a field, part of your potential would be unused. Your education would be only half-complete which could jeopardize your options to get a data science/AI job.


Conclusion

Several reasons explain why someone may not be well-suited to work in some discipline. These three factors – intelligence, education, and knowledge – can be extended to every field. Data science, despite its attractiveness and transversality, isn’t exempt from this situation.

Some people won’t have the innate potential necessary to thrive in the field. Others won’t enjoy a good education because of different reasons, so their potential would remain unfulfilled. Others would dismiss knowledge that is important for the field ending up with unused potential. Even people with intelligence, education, and knowledge may simply decide they don’t like to work at a technical job. Whatever the case, it’s difficult to refute that some people are better suited for data science than others.

One consequence that arises from this situation is that the group of people who would love to work in these fields doesn’t fully overlap with the group of people well-suited to work in them. However, this article isn’t meant to discourage people from trying to find their place in data science or AI. It’s meant to describe those subtle factors that, even if we don’t consider so often, also influence the outcomes. With a more complete picture in mind, we can better tackle the challenges between us and our goals.

This discussion is also important in the sense that data science will become even more ubiquitous in the future. Its transversality could make it a necessity to have tech-related skills in many jobs that now feel too far to be affected. Given the arguments in this article, if data science and AI end up replacing workers in the near future, saying "we can help people relocate to new jobs created by AI" won’t be an achievable policy.


Travel to the future with me for more content on AI, philosophy, and the cognitive sciences! Also, feel free to ask in the comments or reach out on LinkedIn or Twitter! 🙂


Recommended reading

Four Mistakes to Avoid While Learning AI

5 Reasons Why I Didn’t Take a Master’s Degree in AI/ML/DL


Related Articles