5 ways to successfully execute your data science innovation projects

Charlotte Tu
Towards Data Science
5 min readJul 22, 2022

--

Photo by Kvalifik on Unsplash

In my experience, most ‘innovation’ effort in established companies is focused on coming up with ideas. Take hackathons for example — the best ones bring together different parts of the business to come up with great ideas to solve customer problems. But many hackathons have you been part of over your career? How many of the ideas you came up went anywhere? People want to find meaning in their work, and companies that can’t offer that will struggle to keep people.

I recently read Beyond the Idea by Vijay Govindarajan and Christ Trimble. They argue that organisations should shift their energy from coming up with ideas to execution. In this article, I’ll talk about what you can do to make sure you can execute your data science and AI innovation projects.

1. If it’s complicated, create a separate dedicated team

Lots of organisations aspire to have a ‘culture of innovation’. To them that means everyone can come up with ideas and deliver them. The reality is people have a full-time job to execute, which contributes to the P&L today. That means that you are asking people to ‘squeeze it in’. When pressure comes to hit a quarterly revenue target, innovation typically falls to the bottom of the pile and progress can be slow. Some companies can make it work; Toyota are credited with being one of the first companies to create a culture of continuous improvement. It’s important to make it very visible who is contributing, and who isn’t in this model. But these projects are always limited in size. You cannot deliver large scale innovation projects on the side of someone’s desk.

For ambitious, complex innovations, the activity must be separated from BAU operations. It’s not to say that people with BAU roles can’t be involved (in fact they should be because provide valuable input on what customers want and need), but there needs to be a dedicated team, with a specific plan.

This is the mistake I see most often; companies ask people to deliver innovation projects as a side of desk activity, and inevitably projects die when BAU priorities take over.

2. Recruit the right people.

It sounds so obvious — but without this we might as well not talk about the other ones. Time pressure is inevitable with these projects, and it is easy to make the decision to move transition existing staff to move ‘at pace’. Go external if you don’t have the skills in house, even if it takes longer to find the people. Innovation projects with too many ‘insiders will default to the BAU ways of working and the existing organisation model. More than just the skills, individuals that are recruited externally will naturally challenge the existing ways of working and it is easier for them to set up new ways of working because they don’t know what the old ones are. Of course, the team can’t be fully external because they don’t know the organisation, but Govindarajan and Trimble argue that externals shouldn’t be less than one in four.

3. Appoint the right leader

The background of the leader of the innovation project will significantly affect the focus and decision that are made. In the data science world, that normally means deciding between technical / business focus. It is impossible to answer what is right in this article as it depends on the project but there is one common trap to avoid. Whatever you do, don’t ask someone to split their time between leading the BAU work and the innovation project. It’s a tempting thing to do, but inevitably what works to run BAU won’t work to deliver innovation.

4. Create new metrics / policies

There is little point setting up a separate team if they are measured, evaluated, and rewarded in the same way as if they were running BAU operations. I’ve seen organisations that didn’t fall in the first trap, and set up a dedicated team to design, build a prototype some new software. But the lead of the project was still measured on their £2m sales target. A few years down the line, the software is generating far more than this, but for the first few years this was clearly unachievable. Given variable pay was so closely linked to the sales target, this person was personally far worse off taking leadership of this project.

On the whole, leaders of innovation projects should be measured more qualitatively. It tends to be more about the process they go through, rather than the outcomes they deliver. Key things to look for include learning quickly, building a series of experiment, and using the results of these experiments to adjust the plan (rather than gut feel).

5. Don’t annoy BAU operations

There is a temptation as a dedicated team to build a culture that has an ‘us versus them’ mentality. Inevitably this creates tension and is unproductive. The BAU operations are the ones generating the revenue and so pay for the innovation, and the innovation is very rarely completely unrelated to what the business does today. From a personal perspective, if you were the leader of the BAU operation, and a colleague kept referring to ‘breaking the rules’ it would be hard not to take it personally. Inevitably, when it comes down to a fight, the manager of the BAU operation will always win. Typically, they are more senior, they have a well-connected network, and their pressures are to deliver revenue now. The most effective innovation leaders realise that to be successful they need to partner with BAU operations and create a culture where it is unacceptable to be dismissive of the existing profit centre.

Most of these are obvious, but when it comes to execution, people let the pressure to show progress get to them and they cave into what is easy at the time. If you only do one thing, carve out people to do the project on a full-time basis, this is a great start!

References

Govindarajan, V. and Trimble, C. (2013) Beyond the idea: simple, powerful rules for successful innovation.

_________________

I like to write about data science for business users and I’m passionate about using data to deliver tangible business benefits.

You can connect with me on LinkedIn and follow me on Medium to stay up to date with my latest articles.

--

--

Analytics @ HSBC, ex PwC. Passionate about making technical content easy to understand, visualisation and delivering commercial outcomes. All views are my own