Thoughts and Theory
The Inferiority of Complexity
Why Fast and Frugal Heuristics do it Better
Seventy years after Nobel Prize winner Harry Markowitz’s development of modern portfolio theory, mean-variance optimisation remains a hip cornerstone of portfolio selection methods and a key component of the fund manager’s statistical toolbox. For contemporary investors seeking to minimise volatility and increase their returns, the standard combination of multiple regression and mean-variance optimisation is the order of the day. There’s comfort in complexity and the menagerie of statistical strategies developed in the world’s top business schools provide a refuge to fund managers seeking the reassurance of rationality and the logic of statistics. But are the crystal balls of investors clouded by complexity?
Enter, fast and frugal heuristics. Despite the convincing calculations, the infallible formulae and the rationale behind common but complex statistical methods, fast and frugal heuristics offer a refreshing alternative to such, computationally expensive, strategies. Fast, because their use enables decisions to be made quickly and frugal because decisions are often made by ignoring all but some of the available information (predictors). By reducing calculations to simple and transparent methods, heuristics can make decisions quicker and with less information than complex methods.
Heuristics aren’t new and it’s well known that we frequently rely on them in our decision making, an assumed consequence of our bounded rationality, i.e. our limited capacity to make rational decisions due to intrinsic and situational constraints. The use of heuristics in humans (and other animals) is often instinctive and tolerated as a second-best strategy, a necessary trade-off between time and accuracy. However, simple heuristics have found predictive success in domains including sports, medicine, finance and politics. In 2006, Scheibehenne and Bröder found amateurs using the recognition heuristic were more successful at predicting the outcomes of Wimbledon than the experts, similarly, in 1997 Green and Mehr found the use of fast and frugal trees by doctors in A&E more accurately predicted heart attacks than complex alternatives.
Nowhere is the debate on the use of heuristics more contentious than between the works of Gigerenzer, and Kahneman and Tversky. Here, bias in human decision making is considered either friend or foe, a contributing factor or a significant irrational influence, mitigated only through the employ of statistics and logic.
Before we continue, I’ll refer back to the statistically-savvy investor to introduce a familiar trade-off heuristic: the 1/N rule. Known as the equality heuristic, it can be applied in situations of resource allocation, distributing resources equally across N options, hence, 1/N. I’ve represented this with pizza and plates in Figure 1. Ostensibly, this might seem an irrational method but considered against 14 alternative investment strategies in a study by DeMigeul et al. using the Sharpe ratio, non performed consistently better than 1/N. In fact, only Markowitz’s mean-variance succeeded when evaluated with 250 years of training data for 25 assets. This isn’t to say, however, that 1/N should be the go-to, only that there are environments and circumstances in which simple heuristics can outperform complex strategies. Indeed, Markowitz himself relied on this simple heuristic for his own investments. N.B. no comfort of complexity here.
Why is it that simple heuristics can outperform more complex strategies? The answer lies, not necessarily in the heuristics themselves, but in the nature of the environment in which they’re employed.
Rationality in Situations of Uncertainty
Neoclassical economics is underpinned by rationality. In this paradigm of rationality, decisions are made through consideration of all relevant outcomes, their consequences and probabilities. This is the familiar and logical world of the business school, a world of risk, where the future is certain and optimisation is king. In a world of risk, heuristics are second-best.
Outside of the casino, situations of perfect knowledge and certainty are rare. In 1954, Savage described this situational contrast in small and large worlds. In small worlds, outcomes are knowable and explainable — think roulette; in large worlds, the future is uncertain and decisions must be made in spite of information constraints. In Incerto, Nassim Nicholas Taleb posited the application of decision strategies, curated in our small world academic laboratories to large world situations as the ludic fallacy. Describing the impossibility of fulfilling the requirements of complex predictive strategies and highlighting the dangers posed by unknowns to such models. In large worlds, the basis for rational decision making is unrealised.
So, it’s large worlds that are home to heuristics. Where fewer cues and fewer data can lead to better results: heuristics are not second-best strategies.
Prediction: Where Less is More
Simple heuristics violate our ideas of rationality. We’ve seen, at least in one situation, that something simpler can do something better and that the most complex strategy isn’t necessarily the best. To explore this further and to widen our understanding of the situations in which heuristics are favoured decision strategies, I’ve attempted to replicate the 1999 findings of psychological celebrities Gigerenzer and Todd, in their book: Simple Heuristics That Make Us Smart. But before I get into it, bear in mind a few of conditions for heuristic(al) success:
- heuristics work well when there’s a lot of predictive uncertainty;
- when there are many alternatives; and
- when learning opportunities are few.
These conditions lead, not only to the success of simple methods but to the downfall of many complex ones. In complex strategies, predictive uncertainty means there’s little possibility for accurate assumptions (or rather, a greater possibility for inaccurate ones), vast alternatives means there’s a need to estimate more parameters (and in doing so create more errors) and finally, fewer learning opportunities means poor generalisation.
Let’s briefly run through model complexity. Generally, complexity can be considered as the number of free parameters within a model; they’re the variables we know, quantify and estimate weights for, depending on their influence. Complex models with a high number of variables require more estimations and more estimations generate more opportunity for errors (especially if the data we’re using is a bit large world noisy). As a consequence, these models tend to overfit — a product of the bias-variance dilemma. If you’re not familiar, you can read about the trade-off that plagues machine learning here. For now, just know that overfitting usually equals poor predicting, producing the kind of upside-down U-shaped function in Figure 2. Frugal models benefit from their high bias and lack of variance, producing better predictions and weeding out the signal in the noise.
The speed, accuracy and frugality of simple heuristics allow the creation of robust predictive strategies. For this example, we’ll look at the robustness of four strategies in fitting and predicting the population of 83 German cities based on nine binary predictors (Figure 3).
The strategies considered include multiple regression, a typical statistical method, as well as heuristics that learn fewer variables and search fewer cues:
Multiple regression is the most computationally expensive strategy, we’ll consider that our “complex model”. It creates a function by finding the lowest squared distance between data points and a hyperplane (think: line).
Take-the-best is a fast and frugal heuristic. It ranks cues by importance and chooses the best (the cue which discriminates between all others most successfully) — a non-compensatory strategy. AKA one-good-reason.
Dawes’ rule, like regression but faster. Rather than find optimal parameter weights it assigns only a +1 or -1. Not frugal.
Minimalist, like take-the-best but without the initial ranking. Very fast; frugal.
The results are (un)surprising. Regression, the most exhaustive method, searched through all the cues, assigning weights and creating the best fit with an accuracy of 75%. In prediction, regression faired second-worst, a few percent above the minimalist method which chose a cue at random and stuck with it. Two heuristics: take-the-best and Dawes’ rule came out on top, with a prediction accuracy of around 72% for take-the-best. But what’s the fuss? Regression still hits a respectable 71% in prediction.
Computationally intensive strategies are expensive and heuristics have been given a bad rep for favouring time over accuracy. These results show that’s not always true. In some circumstances, you can make more accurate predictions by putting in less effort. And in cases where simple strategies are just as successful as complex ones, it makes sense to save time (and money!).
Another significant environmental factor that promotes the use of heuristics is one in which there are few learning opportunities. In the real world, human decisions are influenced by their experience, what they’ve learnt has a bearing on their actions. Indeed, this can mean many things in a variety of situations and is often considered an unwanted bias and an exemplar of the fallibility and limitations of human decision making but for now, let’s consider learning opportunities as the size of our dataset — that’s how much data we’ve got to work with in order to build our models and produce predictions.
To see how learning opportunities impact the predictive accuracy of models, I’ve again tried to reproduce the findings of Gigerenzer and Todd, using the same models as before, assessing their accuracy across various sizes of training data (Figure 4).
Again, no surprises. What we’re seeing here is the less is more effect, where more accurate decisions are made with fewer data and less computation, rather than more.
Our complex model loses out to at least one simple heuristic until training data goes beyond 80% of our available data. This is unrealistic. In a world of risk, we know the sample space; in a world of uncertainty, we know very little. It’s here where the frugality of heuristics comes into its own and accurate decisions can be made quickly and easily with very limited information. Notably, take-the-best (our A* heuristic), only manages a 6-point increase in prediction performance given a nine-fold increase in the amount of available data.
The poor predictive accuracy of complex models given situations of uncertainty contrasts with presumed superiority of rationality in the work of academics such as Kahneman, who have led the (anti)heuristic debate, claiming their biases and frugality as the product of our irrationality and a limited System 1, rather than championing their circumstantial successes. Of course, contemporary computational advances and research into heuristics have allowed for strategies previously only understood as instinctive and naïve to be quantified and analysed. There are situations in which complex strategies are incredibly successful and others in which overlooking the simpler and faster implementation of heuristics is a costly, if not potentially harmful, oversight. These situations can be better understood through the concept of ecological rationality.
Ecological Rationality
Better decisions can be made with less information, fewer resources and less time. Since the large world is the real world, the effectiveness of heuristics in the face of uncertainty makes them essential decision-making tools. But, the use of heuristics should be employed in environments where they’re appropriate for a successful outcome, in other words, environments in which their use is ecologically rational.
Ecological rationality can help us decide whether it’s best to opt for a complex statistical strategy over a fast and frugal one and vice versa. Beautifully explained here by Luan, Reb and Gigerenzer in the context of managerial decision making, their study finds that heuristics are not second best to rational strategies.
Where not all consequences, alternatives and probabilities are known; given minimal data and vast alternatives, the robustness of fast and frugal heuristics offers an ecologically rational choice. An example with which we’re all comfortable is the allocation of resources, here we’ve seen that 1/N is ecologically rational due to predictive uncertainty, a large N and a lack of meaningful learning opportunities.
Complex strategies are long-sufferers of overfitting. In situations of personnel selection, resource allocation, emergency medicine, politics and sports predictions, heuristics are effective decision-making tools. These are all situations of uncertainty, often masquerading as risk, where prediction is more important than hindsight and in some cases, where the transparency and memorability of heuristics aid human decision making in a way unrivalled by computationally expensive methods.
Notes, References and Recommended Reading
I highly recommend two of my favourite books as essential reading for those interested in risk, decision science and statistics: The Black Swan (2008) by Nassim Nicholas Taleb and Thinking, Fast and Slow (2013) by Danny Kahneman.
Gerd Gigerenzer has led some incredible research into heuristics but it’s pretty heavy so here’s a link to a TED Talk. You’ll see he discusses 1/N and reviews the simple heuristics that I’ve written about.
References to resources I couldn’t find a link for:
