Data Scientist to Data Strategist
On the most important job nearly no one has
--
The last six months were, in many ways, some of the best of my professional life. My company was able to do work for huge businesses, my book proposal was accepted by a publisher and, thanks to the new methodologies I’d developed for doing my work, I was achieving better results for my clients than ever before.
I had been trying to reach that point for years. I’d been aiming towards building important things for clients with big budgets and sharpening my technical chops along the way.
But when I got there I found out that something wasn’t right.
One client, an international telemarketing business, was thrilled with my efforts. They took me to lunch, praised the work, and referred me to others in their industry. And yet, when it came time to deploy the system we’d built, they backed off and decided to put the deployment on hold.
We built a risk analysis toolkit for another client and followed their specification exactly. But when it got into the users’ hands I found out that no one was using the recommendation engine that had been a core part of the proposal, a key ingredient driving the entire project.
It might seem silly to complain about happy customers who pay you on time. But if you’re trying to grow and develop, knowing your hard work is sitting on a shelf leaves a sour taste regardless of how well the project went.
For the past couple of months, I’ve been trying to figure out what happened. Why were my clients happy on the surface but not using the things I’d built for them? Why did I suddenly feel that all those hours spent honing technical skills had been wasted? Where do I go from here?
The mistake I had been making the entire time was this:
I was not having the right conversation with my clients.
Shiny New Thing Syndrome affects everyone. Businesses want to outdo their competition, have bragging rights, be the yardstick used to measure innovation. Consulting firms, similarly, want to add as many logos as possible to their testimonial page and build up their repertoire of case studies.
Chasing new projects, new clients, and new logos means doubling down on what you know. It means becoming automatic with the deployment of your skills.
Now there’s certainly something to be said for expertise, but when you apply them blindly you are doing your clients a disservice.
When I got a lead in the past, I’d ask about budget, goals, and timeline. The usual things that helped me determine if we could work together. As soon as that was covered we’d talk about algorithms, AlphaZero, self-driving cars, potential future projects. Basically, we’d discuss anything that upped the excitement level.
But there was something we never discussed: why.
Why do you want to do this project? Why now? Why do you want to do it using machine learning? Why not use a spreadsheet or some simpler mathematics? Why do you need me?
I know why I didn’t ask these questions.
I was afraid that it would kill the enthusiasm. I was afraid that it would lead me into a career of doing simpler, less sexy projects. I was afraid that my technical skills would go to waste.
Growing pains hurt, it’s right there in the name. And it’s taken me some time to come to terms with the fact that I’d need to put away technical solutions in order to solve actual problems.
For the past few weeks I’ve talked to clients in a different way. I’m having the conversations I should’ve been having all along. It’s like I’ve looked up from the screen and made eye contact with them for the first time.
By asking the deeper questions, I’ve taken them further back in the conversation to the point before they made up their mind to do one particular project, one particular way.
This means that I get to understand what they really want.
Nobody wants a complex, expensive, difficult-to-maintain machine learning system for the sake of it. They want to reach more customers (or lose less), make staff happier by reducing the amount of menial activities they have to do, or automate what they can to improve efficiency and the bottom line.
Sometimes these things need machine learning and sometimes they don’t.
Even if machine learning is the best way to attack a particular problem, the chances are slim that a client has picked the right project to start on, picked the right metrics to measure, or allocated the right amount of resources.
Machine learning solutions don’t start in a text editor or a Jupyter notebook— they start in a conference room with senior executives pouring over the annual report. You forget this at your client’s or employer’s peril.
By being steadfast in this new approach of asking why, I’ve been able to have entirely new conversations — it has completely changed the way my clients view me. Now I’m someone who can be trusted to tell the truth. Now they know I’m not only interested in doing the next new thing.
Data scientists are going to be key employees in organisations for years to come and that’s not up for debate. But there’s a vacancy that hasn’t been filled yet and that’s the data strategist.
Companies need someone who’s been through the ringer, seen analytics succeed, seen it fail. Someone who knows when to kill a project. Someone who knows what the cheaper alternatives are when everyone else is gushing about the potential of this or that algorithm. They need someone who’s aligned with the business at the expense of their own technical expertise, at the expensive of not playing with the shiniest, newest things.
We data scientists owe it to our employers and clients to solve their actual problems, and in order to do that we’re going to have to grow in to data strategists.
When all you have is a hammer, everything looks like a nail. So data scientists, please think strategically, think honestly, and don’t let your eyes be drawn to the shiny things. Ask why.
