The world’s leading publication for data science, AI, and ML professionals.

Why data scientists and business executives struggle to work together

How to bridge the gap between business executives and data scientists to overcome challenges and move towards data- and business-literate…

Photo by Vladimir Proskurovskiy on Unsplash
Photo by Vladimir Proskurovskiy on Unsplash

When you think about data scientists*, most likely you are imagining someone working in a tech company that has data solutions as their core, like software providers. Perhaps, you also assume that the majority of the employees are familiar with the technical limitations and potentials of a data-driven business. As a result, you expect a seamless work-relationship between data scientists and business executives.

However, there is a growing struggle between data professionals and business executives. The reasons for this impasse are two-fold. First, the demand for data scientists and Artificial Intelligence skills are quickly spreading beyond the tech industry [1]. Second, although, non-tech companies recognise the benefits of data-driven decisions, they have little – if any – experience managing tech projects. Combining the urge for competitive advantage in the market with the lack of developing analytical projects create a massive gap between business executives and data-professionals.

As a data scientist you find yourself on the tech-analytical side of the tug of war. Also, it might have crossed your mind that business executives are clueless when managing their companies. Conversely, if you are a business executive, then you often listen to advice from software consultants or salespeople outside the company. And, unsurprisingly, those experts almost always promise innovative solutions that your team cannot deliver. You wonder, frustrated, why your team usually say that a simple is just ‘not doable.’ Well, here is where the gap begins to surface.

Finding the Gap

Usually, the gap appears when business executives have an idea that sounds simple, but he or she is unaware that it is an incredibly complex Data Science problem. Unlike companies with a few products ( consumer good), other companies (financial sector) might have thousands of products, with millions of transactions. Also, instead of a product with a stable price for a large part of the year (a bottle of Coke), you might find different prices for each customer negotiated every minute (sports betting). In such companies, data scientists trying to build a model that allows business executives to estimate sales volume is arduous job. How so? Because of the availability, volume, accuracy of data or the level of complexity of the resulting models. It is nothing like P&L on a Excel spreadsheet.

The gap increases as the idea moves from top executives down to someone with enough programming expertise to say: "Wait, that’s not doable." Then, their managers have to start the difficult task of taking the bad news uphill. The back and forth between the data team and business executives might go on for a long time. Inevitably, like any other top-down decision, there are consequences. Either the initial idea will change significantly, or the data team will take ages to develop something that will not work as initially thought. Either way, the company loses, and you (the data professional) gets frustrated. Unfortunately, this tends to happen frequently when a company in one industry tries to copy initiatives from companies in other sectors without accounting for the differences in their operating models. So, if you work in one of those companies, you might need a reality check and manage your expectations.

The Reality

Let’s be honest: money is the measure for most things in business. Hopefully, this is not a surprise to you. Business executives and entrepreneurs are tirelessly developing market strategies and making decisions at a fast pace, whereby each decision must be profitable. From a business point of view, if an idea has the potential to achieve the company’s goal, then it might sufficient to implement a top-down decision – even if apparently it is not doable. Let me give you an example of how innovative business executives think:

Tesla and SpaceX CEO, Elon Musk, sets nearly impossible goals for his employees. One of Elon’s engineers [2] has said:

"He will pick the most aggressive time schedule imaginable assuming everything goes right, and then accelerate it by assuming that everyone can work harder."

Although not everyone is like Elon Musk, setting high bars is part of the job description of business executives across all companies. One can argue that CEOs like Elon Musk or Steve Jobs are an exception. Fair point, but there is a reason for their success: they are tech-literate professionals who are aware of what their team can and cannot do. Acknowledging your flaws and limitations is the first step to change.

Photo by OnInnovation on Flickr
Photo by OnInnovation on Flickr

The New Mindset

Companies that are more mature in their analytical evolution have a similar philosophy to the one shown by Elon Musk [3]. Senior executives are increasingly becoming better educated about data science and artificial intelligence skills. Also, some companies have moved from a vertical binary division (business vs tech) to a horizontal and gradient way of working with middle management professionals capable of translating upwards the challenges faced by the data team. The evolution to a new mindset may bring the sought-after competitive advantage, but in different forms:

  1. New roles, such as VP of Data, will become the norm. As a consequence, initial ‘simple’ ideas will either be intercepted or fined tuned as challenge or an innovation project. Companies will not waste resources a corporate tug of war.
  2. Outside experts and tech partners will no longer jeopardise the internal work relationship between senior executives, including the marketing department, and the data team. Eventually, business executives will be capable of more in-depth and productive discussions with tech suppliers. In turn, outside partners will add value to both the business and the data team.

How data scientists could bridge the gap

If you are a data scientist, especially if in a career transition, then you might want to think about expanding your horizons by acquiring business skills. Why? In the same way, business executives will become more data- and tech-literate; perhaps it is worth learning about business and marketing strategy. In doing so, you will not only understand business executives better but also be able to create strategic data products relevant to your niche. Ironically, by acquiring those business skills you will have a competitive advantage in the job market and become a more robust professional. You will be able to adapt to any business environment, whether in a team meeting, in a conversation with the company’s CEO, or even when setting up your own business.

Conclusion

Business executive and data professionals have complementary skills. Data science and AI skills have been growing and expanding to most industries. However, data professionals and business executives are struggling to create a productive work relationship, which creates a massive gap. The issue lies in the less data educated business leaders who want to implement simple ideas, which from a data science perspective, are just ‘not doable.’ Yet, making the business profitable is what drives every company, and employees have to simply adapt to the business environment. That said, there is an opportunity for companies to create middle-management positions between the company’s extremes. More importantly, data scientists could fill this gap by acquiring business knowledge and evolving into a data-driven management position. So, what is the key takeaway? It is not either data or business, but data and business.

  • Throughout the text I used data scientists and data professionals as a generic term for the broad category of professionals working with data. Those professions include, but are not limited to, data analysts, Machine Learning engineers, software engineers and artificial intelligence professionals.

Since you are here, you might enjoy reading my other article on TDS:

Are bootcamps worth it if you are switching career to Data Science?


References:

[1] https://economicgraph.linkedin.com/blog/how-artificial-intelligence-is-already-impacting-todays-jobs

[2] https://www.businessinsider.com/elon-musk-sets-nearly-impossible-goals-for-spacex-2015-5?r=US&IR=T

[3] https://www.forbes.com/sites/bryancollinseurope/2018/05/21/elon-musk/?sh=274434e651d4


Related Articles