How to make data driven decisions like a boss

Published in

Towards Data Science

8 min readJun 7, 2017

In theory, there is no difference between theory and practice. But in practice, there is. ― Yogi Berra

Decision-making can be hard. The number of decisions you need to make everyday can be overwhelming. Research shows that the more decisions we make in a given day, the worse we get at decision-making (decision fatigue). Sometimes you are making really, really hard decisions, where it is hard to choose one option over another even using all the data you have at your disposal, in the midst of this fatigue.

Decision Theory is a combination of logic, game theory and probability that seeks to formalize decisions into structured problems you can more easily solve. Great, but why should you care? Because it provides a number of common frameworks and tools that you can use to ensure you are consistent in your approach to decisions. The better you are at using these frameworks and tools, the better you will get at decision making — and the less likely you are to be overwhelmed and fall victim to decision fatigue.

Decision Theory is a large field, so we will only cover the basics you need to get started. Specifically we will cover:

Lets get started by modeling decisions to make them easier to evaluate.

Outlier monitors your business data and notifies you when unexpected changes occur. We help Marketing/Growth & Product teams drive more value from their business data. Schedule a demo today.
- Outlier was the Strata+Hadoop World 2017 Audience Award Winner.

Decision Models

Visualizing Decisions

What is a decision? Simply put, it is a choice between a number of different options, each of which lead to some expected outcome. You can visualize a decision using what I call a Decision Model which are really just Flow charts. You have probably used Flow charts in the past to capture processes, which is exactly what we need for our decision making process! For example, the following is a very generic decision model:

A simple flow diagram describing decisions

Decision models help you capture all the information you need in order to make an optimal decision:

Enumeration of all possible choices.
Understanding of the result of each choice.
Understanding of the eventual outcomes.

Decision models make it easy to visualize all three of these. For example, let’s say we need to decide if we should hire another salesperson for our sales team. If we do hire the salesperson, it will cost some money (salary) in the short-term but likely lead to more sales in 6 months. That decision may look as follows:

A hypothetical flow evaluating the hire of a sales person

As you can see, a decision model is just an easy way to make sure you understand all the options and outcomes for a decision by forcing you to write it down. You would never do this for every decision, but for the most important ones it’s a critical tool.

Next, we’ll talk about how to understand outcomes when there is uncertainty involved, which is almost always the case in real life!

Expected Value Analysis

Understanding Outcomes

In discussing Decision Models yesterday, we broke down how to think about a decisions and the options you can choose between. In many ways, even more important than the options you choose between are the eventual outcomes of those choices.

For example, let’s say you are trying to decide if you should buy a lottery ticket. We can model the decision as follows:

Flow diagram for expected value analysis

Looking at the decision there, obviously we should buy the ticket! However, what we haven’t captured is that the probability of winning is not 100%. In fact, for most lotteries the probability of winning is extraordinarily small. Let us assume the probability of winning this lottery is 1 in 175 million (typical for the Powerball). How then do we measure the value of that potential outcome?

Expected values are a way of evaluating outcomes that are subject to probability (also known as random variables). The expected value allows you to take into account the likelihood of event when quantifying it, and compare it with other events of differing probabilities.

To calculate an expected value, you multiple the probability of the event by the value of the event. In this case, we multiply the value of winning ($100M) with the probability of winning:

In simple terms, the payoff for winning is huge but the chances of winning are tiny so the expected value of buying a ticket is only 57 cents.

We can now use this in our decision model to make the decision clear:

Now the choice is clear, we are much better to not buy the ticket and save our $1 than buy the ticket and lose $0.43! This is the power of expected values, they allow us to quickly and easily account for probabilities when comparing options.

Of course, most decisions will include many different probabilities. Next, we’ll cover multi-stage decisions where one decision may lead to many others, and bring together what we’ve discussed so far to make some actual decisions!

Decision Trees

Few decisions in life, or in business, stand alone. One decision leads to more decisions which, in turn, lead to even more decisions. When making a decision you need to consider those later decisions if you are to make an optimal choice.

A decision tree is an extension of the decision model we discussed previously, which allows you to understand the relationships and eventual outcomes from connected decisions. The only difference between a decision model and a decision tree is that in a decision tree, the outcome of a choice can be another decision!

As an example, let us revisit our decision about whether to hire a salesperson. Instead of thinking about it as a decision to hire a salesperson, let us think about it as a decision whether to hire a salesperson now or later. This changes the decision into a tree as you can see below:

Taking the decision tree to the next level with dependencies

If we decide not to hire the salesperson now, we open ourselves to the option to hire them later. This has both costs and benefits that you can begin to analyze now that you have the decision tree mapped out.

This simple example captures many of the characteristics of real world decisions and things you should consider when building decision trees:

Just because there are a large number of decisions, there might only be a small number of possible outcomes. In many real world circumstances there are only a few possible outcomes to any line of decision making and this can greatly simplify your process. By working backwards from the preferred outcomes you can identify the decision paths most likely to help you reach it.
Depending on the choice we make in the first decision, various other decisions either become available or are lost. This is an important factor to consider in decision making, as some choices may preserve more choices in the future and hence increase the flexibility you’ll have for future decisions.
Many of the branches of your decision tree will also introduce uncertainty, as some decisions and outcomes will rely on certain probabilities. Here, using expected values is very powerful as you can calculate the expected value of a choice even if the true outcome is many decisions removed!

The larger your decision tree the harder it will get to deal with the growing number of probabilities it may introduce. Tomorrow we’ll review a useful tool to cut through that complexity and make those decisions a little easier.

Simulating Decisions

Gambling Can Be Good

Even with the tools we have discussed this week, it can still be hard to make tough decisions. If your decision tree has a lot of uncertainty or your expected values are hard to predict even these frameworks can offer little aid.

The good news is that you can cheat! You can run through the same decision thousands of times and see how it turns out before you make your decision for real. That process is called a Monte Carlo simulation, which uses software to simulate making the same decision many times and, using the probabilities you provide, determines the outcomes. By studying the outcomes across so many tests, you can understand the most likely outcomes and use that as part of your decision making.

For example, let us return to our decision about whether to buy a lottery ticket. Remember, the ticket cost $1 and the payout for winning is $100M, but the chances of winning are only 1 out of 175M. What if that same ticket also provided a 1 in 20 chance of winning $5 and a 1 in 50 chance of winning $20? How do we make the decision then?

For this simple of an example, we could calculate the expected values directly, but for fun, I ran a Monte Carlo simulation, which generated the following winning outcomes:

I ran the simulation 500 times, of which I made nothing 467 times, $5 15 times, $20 16 times and, sadly, $100M 0 times. The mean outcome was winning $0.79 — still less than the price of the ticket!

However, considering the probabilities involved in this simulation 500 times is probably not enough to have a consistent result. As with any probability simulation, the more times you run it the more your resulting distribution of results will reflect the true probabilities so always run it as many times as possible. I re-ran this same simulation another 500 times and got a mean value of $0.54 which is vastly different! This highlights the dangers of this kind of approach, where you need to use the results as guidance but not fact since there is so much variability in the results.

To Review: We’ve covered the basics of decision theory, including how to visualize decisions to ensure you capture all possible choices and outcomes (decision models and trees) and account for uncertainty in the outcomes (expected values). I hope this gives you a start in approaching your decisions in a systematic way that will help you make better decisions!