Analysts: stop measuring everything else, and start measuring the value of your data projects!!!
TL;DR
Measuring the business value of your project is dead simple, and has so many benefits, but it seems like nobody thinks to do it. So here’s one way how to do it.
I’ve had the good fortune to work in the data and analytics space for 5 years now —in two fortune 500s, and now in a startup.
I’ve helped organise meetups, I’ve attended multiple conferences, I’ve watched plenty of online courses, and I’ve learned from and worked alongside the unicorns…
Based on all this experience, I would argue that analytics as a discipline has a blind spot on how it measures return on investment (ROI). The irony is that it obsesses on how to measure everything else!
This was highlighted in a recent panel on “Making the right decisions on what to automate” at the Sydney AI and Machine Learning Summit. Very smart, accomplished people spent 40 minutes talking through all the technical concerns in choosing a problem, but didn’t even spend a second on considering how you’d size the business value of solving the problem — and whether your project is worth the cost.
And I get it — an analyst or data scientist would much rather be tuning a model than concocting a business case. But showing the value of your work is dead simple, and it is the key to building trust with senior management and securing even more resources for the next project.
So in the interest of paying forwards all the help I’ve had on this topic, this article contains my method in calculating the business value of a data project.
I’ve broken the method down into an equation, some theory, and some simple examples. It is hoped these examples will demonstrate how easy this method is to practice, and how this method can accommodate a tremendously wide range of projects. I’m super happy to take a crack at your particular example in the comments — in the interests of time, I’ve kept it to two, but I apply this daily and am literally staring at a airtable of ~140 business value calcs that I use to justify my existence.
Finally, this thinking SELLS. It worked for Oracle Direct selling hundred thousand dollar contracts into enterprises. It worked for me at Salesforce in securing hundreds of thousands in budget. It’s working at Jayride… touch wood! And it will work for you — or you can flame me to oblivion.
EQUATION:
V = (S + M + R + O) — C
FACTORS:
V = Business Value
S = Money Saved
M = Money Made
R = Risk Reduced
O = Opportunity Captured
C = Sum total of Project Costs
THEORY:
Data Science, BI, or anything analytics is a function within a business. A business function shows value to management through Saving Money, Making Money, Reducing Risk, or helping to Capture Opportunity.
The net Business Value, is simply the sum of all these factors, minus the Cost of the project. You also divide by the Cost of the project, to give an “Return on Investment” ratio that can be compared by management (and yourself) to other initiatives.
How do you measure all these factors?
For Money Saved (S), it’s dead simple — get an estimate of how much time you’re saving somebody, and know the average value of an hour of human help. A good rule of thumb is $50 per hour — so for x hours saved, you’ve brought in S= $50x in Money Saved.
For Money Made (M), it’s pretty simple — you benchmark the revenue performance of the thing you’re helping out before your project (b), and measure the revenue performance of the thing after your project (a). The difference between performance, M = a-b, is the Money Made from your project. If you’re spinning up an entirely new way of making money, you don’t have to worry about benchmarking(i.e. M = a). Avinish Kaushik provides a comprehensive approach to this kind of measurement here.
For Risk Reduced (R), you estimate the worst case cost (w), the probability of this happening before your project (s), and the probability of this happening after your project (t). As Risk is the multiple of cost and probability, Risk reduced is simply the difference — or R = w(s-t).
Opportunities Seized (O) is simply the inverse of Risk Reduced —you measure the best case benefit (y — for yay! …I’m running out of letters), the probability of this happening before your project (h), and the probability of this happening after your project (e). O = y(e — h).
Pro tip: don’t boil the ocean in estimating the probabilities used for Risk Reduced or Opportunities Seized! Usually a rough gut estimate from a subject matter expert will suffice. And if you really must go deep, find an actuary, or check out Douglas Hubbard’s “applied information economics” (you can get a great overview on less wrong here, or buy his book).
Finally, Costs is simply the Sum total cost of your project’s bill of materials. Don’t forget to estimate the cost of your time. Always know what you cost your business, and make sure the things you work on are worth more than that!
EXAMPLES:
- You’re making a sick dashboard!
This dashboard will help save some executive 10 hours (x) in spreadsheet fiddling per month.
In addition, this visibility will help reduce the risk of people overclaiming your HR system, from 50% (s) to 10% (t), with a worst case cost of $2000 per month (based on historicals).
It will cost you a day’s worth of work to do — or around 8 hours.
In this vase, V=(S+R-C)/C = (10*50+2000*(0.5–0.1)-8*50) = $900/month
Sick!
2. You’re building a groovy model!
This model is all about targeting your customer support teams to call and keep happy high risk customers, and has an opportunity to save the business, best case, $10000 per month in resources. The customer support team is also currently bringing in $3000 per month in cross sells, and you reckon it could be more after your targeting. This will cost you around a week to deliver, and you and the team reckon the likelihood of this benefit after the model is around 50%, and before the model is 0%.
So V=M+O-C = 0+10000*(0.5–0) -40*50 = 3000+ per month.
It’s probably more than this, so don’t forget to measure money made after the project.
Groovy!
In Summary
Measuring the business value of your project is dead simple, and has so many benefits, but it seems like nobody thinks to do it. I’ve given you one way to do it — so do it.
P.S. Special big thanks to the good people at Verge Labs, who spotted me a ticket to the Sydney AI and Machine Learning summit — it’s a great event!
