Can Artificial Intelligence replace Data Scientists?
Steve Urkel from the world-famous ABC sitcom Family Matters was not a Data Scientist because, in the ’90s, we did not call them like that. But the nerdy teenage Urkel is no doubt the archetype of the Data Scientist. In one of the episodes, Urkel built a robot whose intelligence doubled every two minutes. This sounds pretty cool but was a terrible idea. In no time, Urkelbot overthrew its master and locked him in the basement…
McKinsey estimates that about 64–69% of the total time spent on data collection and processing can be automated. As technology advances, many are questioning whether or not A.I. has the potential to work faster and cheaper than data professionals soon.
There are two main kinds of data professionals:
- Data Engineers extract and assemble data from different sources, transform it, clean it, and then load it into a repository in a standardized format.
- Data Scientists take data from that repository to design, build and test advanced models based on machine learning algorithms.
Machine learning is by definition part of A.I. Additionally, A. I can automate many tasks that Data Scientists and Data Engineers perform. For starters, A.I. can be applied to the following functions typically performed by Data Engineers:
- Preparing data, cleansing, checking for correctness, identifying outliers and empty records
- Self-service systems to make data available to teams
- Automating deployment of models into production
Moreover, A.I. can also automate some of the work of Data Scientists in the following ways:
- Detecting relevant prediction features and representing them
- Generate hundreds of thousands of variations of models (for different segments and markets)
- Building basic models through intuitive interfaces
- Detecting obsolescence of models
The combination of A.I. with human problem-solving has empowered, rather than threatened, Data Scientists' jobs.
A.I. can help Data Scientists generate hundreds or thousands of variations of models with different prediction features and create iterative simulations to choose the best variation finally. The best iterations involve both A.I. and Data Scientists. A dynamic, multi-faceted decision process obtained through automation will outperform any single algorithm, no matter how advanced, by automatically testing, iterating, and monitoring data quality, incorporating new data points as they become available, and making it possible to respond wisely to events in real-time.
Additionally, A.I. can assist Data Engineers in preparing raw data, cleansing it, and checking for correctness. This is not something that A.I. can handle entirely yet. It still requires human judgment to turn raw data into insights that make sense for a complex organization. A.I. cannot yet truly understand what specific data means for an organization, its business, and the industry context. A.I. can help automate lower-level steps in data preparation and visualization, leaving Data Scientists to walk decision-makers through what the insights mean.
Lower-level tasks, which Data Engineers typically perform, will be first impacted by A.I. For example, in the ’80s, as computer programming languages advanced, demand for lower-level programmers did indeed decrease. However, demand for developers in general increased as companies adapted to these new languages. The same evolution is happening in analytics, with A.I. automating lower-level tasks. This leads to the transition of more complex, problem-solving tasks to humans. As a result, the combination of A.I. with human problem-solving has empowered, rather than threatened, Data Scientists' jobs.
But Data Scientists and Data Engineers need to adapt. As A.I. automates lower-level data processing, Data Engineers will need to migrate toward data science. Even among Data Scientists, only the youngest has been trained in the more advanced deep learning approaches. As technology advances, Data Scientists' skillset will be rendered useless in 12 to 18 months. Data Professionals will need to either learn new A.I. tools or get left behind.
As technology continues to evolve, the skill set of Data Scientists will be rendered useless in 12 to 18 months. Data Professionals will need to either learn new A.I. tools or get left behind.
Instead of posing a threat to data science jobs, A.I. will likely become knowledgeable assistants to Data Scientists, allowing them to run more complex data simulations than ever before. Analytical skills will soon be required in many more traditional roles. This transition is expected to create a new class of Data scientists — namely Citizen Data Scientists — that bridges the gap between business and strictly analytical functions.
Fortunately, our friend Steve Urkel could finally overthrow the tyrannical Urkelbot, whose intelligence doubled every two minutes, just 22 sitcom minutes after being locked in the basement. Contrary to being a menace to humankind, at the moment when A.I. surpasses human intelligence, Data Scientists are likely to continue working with A.I. or developing newer A.I. systems.
About the author
Pedro URIA RECIO is a thought leader in artificial intelligence, data analytics, and digital marketing. His career has encompassed building, leading, and mentoring diverse high-performing teams, developing marketing and analytics strategy, commercial leadership with P&L ownership, the leadership of transformational programs, and management consulting.
Disclaimer: Opinions in the article do not represent the ones endorsed by the author’s employer.
References
- “4 Reasons Bots won’t Replace Data Scientists Anytime Soon”, Vivian Zhang, NYC Data Science Academy, July 2017
- “Making Data Analytics Work for You — Instead of the Other Way Round”, October 2016, Helen Mayhew, Tamim Saleh, and Simon Williams, McKinsey
- “Automated Machine Learning Won’t Replace Data Scientists”, Jacqueline Berkman, July 2017
- “How AI Fits into Your Data Science Team”, Hilary Mason, Jul 2017
- “The Rise of AI will Force Data Scientists to Evolve or Get Left Behind” Rudina Seseri, Forbes, Jan 2017
- “Why Automation won’t Replace Data Scientists yet”, Jennifer Roubaud, March 2017
- “Will AI Replace the Need for Data Scientists?”, Rob Light, March 2017
- “Big Data Analytics: Role of Automation”, February 2017, Arun Goyal
- “3 Flavors of Predictive Analytics Automation”, May 2017, Fern Halper
- “Where Machines Could Replace Humans and Where They can’t Yet”, July 2016, Michael Chui, James Manyika, and Mehdi Miremadi, McKinsey
