Many large organisations are struggling with how to lead their data scientists effectively. Here’s what I have learned.
The other day I was reading a demography paper on population aging when it occurred to me that there is an analogy between population pyramids and Data Science organizations. A constrictive population pyramid refers to the fact that it is narrowed at the bottom, i.e. there is a lower percentage of younger people. Low percentages of younger people versus older people can cause issues with the dependency ratio of a population. I see some organizations suffer from the same phenomenon, but then in the area of data science, ML and AI skills. Especially large, traditional organizations that try to leverage Data Science and AI seem to be prone to this.
Let me illustrate my point with carpentry. Young people who might or might not have learned some basic carpentry skills at school, typically started working for a more established and experienced carpenter, sharpen their skills and experience, and maybe specialize in some area of carpentry and move from there. After a while some would start their own businesses and maybe hire carpenters fresh from school themselves. Some others would not have the ambition or skills to grow and they would leave the carpentry business or remain at a position that they felt comfortable with. When the old carpenter would retire his or her most experienced and talented carpenter would fill his/her shoes. This situation would be close to a "stationary" pyramid in the sense that you have a relatively large group of low skilled or low experienced carpenters, followed by a smaller group of more skilled or more experienced carpenters, and so on. As the more skilled/experienced group coaches the less skilled/experienced group, after some time, a smaller group would "promote" to the next level.
The situation is somewhat different in Data Science. Thanks to the flexibility and adaptability of our universities – I can’t stress enough how happy I’m about this – many freshly graduated data scientists have an impressive skill set. Yes, they might lack some experience and might be prone to overengineering, but with some proper coaching this can be sharpened out. In fact, because of the fast pace of change in technology, many young data scientists have skills that their older colleagues might lack, simply because it didn’t exist when they were going to college. That’s why it is vital that data scientists keep up with the latest evolutions in their area of expertise. Good organizations allow their data scientists to do that. Good organizations will also make sure that younger data scientists can learn from their older and more experienced colleagues how to apply their skills in a business context. If this happens, data science can be close to the "stationary" pyramid that we saw with carpenters.
<HR class=”wp-block-separator has-alpha-channel-opacity” />Alas, this is not always the case. All too often, the older data scientists are Business Intelligence folks in disguise who have not much to teach to the younger generation (other than bad habits). As long as these people are not given leadership roles they are harmless. The problem starts when people without a proper data science background are being asked to lead a data science team. This seldom works. Yes, we’ve all heard the stories about the dietician or the art historian who turned out to be a stellar data science leader, but these are the exceptions to the rule. I have heard many data scientist complain that their organization has hired a less skilled manager above them. I hear about these complaints from data scientists more often than from folks in other departments. There are many reasons why this happens. To start with, in many (traditional) organizations data science (or ML or AI) is not at the core of the business. They simply hire an army of data scientist because they want to "do something with AI". That team then typically reports to one of the more traditional functions such as IT or Marketing and are thus managed by people from those departments. This will not always lead to a happy marriage.
Another reason is that – we need to be honest about that – not many data scientists are good managers and good data scientists at the same time. It is therefore only logical that companies want to put their (expensive) data scientists in roles that fit them. But, the reality is that this situation leads to a lot of frustration. I personally believe this (partially) explains the high turnaround some data science teams suffer from.
You can wonder why all of this seem to happen more often in data science departments (of traditional companies) than with other functions. For example, I haven’t seen many organizations where the Finance department is populated by financial whizz kids, but the CFO is somebody without a degree or years of experience in Finance But that is exactly what I see in some organizations when it comes to Data Science.
Another example is Legal. Many organizations without a large Legal department still have a Chief Legal Officer who will hire legal help if need be. Yet, this Chief Legal Officer will sit together with the COO, the CMO and other C-suite folks, while the data scientists are, in the best of cases, sitting somewhere deep in the organization of the COO, CMO or CTO. Sure, there are organizations with a Chief Data Officer or a Chief Science Officer, but very often these people are not skilled in (data) science. I was lucky enough that I once reported to the Chief Science Officer of a large company who actually had a scientific background, but I feel that this is an exception to the rule.
So what is the solution then? First, we need to encourage young data scientist to invest in leadership skills, so that the data science leadership roles can be filled by data scientists and not by retiring BI folks.
This is not so easy, because quite often young data scientists find this stuff boring. Moreover, leadership is sometimes confused with politics. Most data scientists I know hate company politics, and rightfully so. It’s often destructive, not in the interest of the company and pretty boring. But, in large organizations, politics are also essential if you want your data science team to remain relevant for the business and generally flourish. I’ve seen a case where the data science leader was feeling above company politics and was focusing on technical stuff only. Initially she gained a lot of respect from her team members. They were impressed by her technical prowess and they liked the fact that she didn’t play the political game. After some time however, her team had become completely irrelevant to the business and people would move from her team to other departments. The same people that cheered her initially were pretty fast in complaining that she didn’t take care of their interests and left. Notice, by the way, that cases like this strengthen the view that data scientists don’t have the management skills to make it in large organizations, which brings us back to square one.
Secondly, organizations better think about who they put in charge of the data science team and how that team fits in the rest of the organization. As I mentioned above, I know many data scientists who are frustrated at the (perceived) lack of skills of folks that are higher in the hierarchy than they are. Sometimes it’s because these data scientists don’t always see the skills that those people do have, then it is a matter of education. But sometimes you just need to hire the right people for the job. Mind you that I’m not saying that they all should have a PhD. This is mostly irrelevant. A long time ago I was part of a small team in a large organization. Incidentally, all of us had a PhD, only the boss didn’t. However, he was also more knowledgeable and more experienced than us, and so nobody questioned his leadership, quite on the contrary.
Let me conclude by repeating that the phenomena I describe above will happen less in organizations that have data science at the very core of their business. I mainly see this happening in large, traditional companies that try to leverage data science and AI. Clearly, in those organizations, data science has a leadership problem or at least a perceived leadership problem. It’s up to data scientists to prove that, when it comes to leadership, data science is more than BI++.
