The base-rate fallacy is the tendency to ignore base-rate information – for instance the average of a population or a sample – and focus on specific information, for instance data pertaining to a certain case, or a small number of cases (Bar-Hillel 1980, Tversky and Kahneman 1982). The base-rate fallacy is sometimes also called base-rate bias or base-rate neglect.
If you play poker and assume different odds than those that apply, you are subject to the base-rate fallacy – and likely to lose. The objective odds are the base rate.
People often think the information they have is more relevant than it actually is.
People often think the information they have is more relevant than it actually is, or they are blind to all the relevant information they do not have. Both situations result in the base-rate fallacy.
"Probability neglect" – a term coined by Sunstein (2002: 62–63) to denote the situation where people overfocus on bad outcomes with small likelihoods, for instance terrorist attacks – is a special case of the base-rate fallacy.
The base-rate fallacy is fed by other biases, for instance uniqueness bias, which results in extreme base-rate neglect, because the case at hand is believed to be unique, wherefore information about other cases is deemed irrelevant.
The inside view, hindsight bias, availability bias, recency bias, WYSIATI bias, overconfidence bias, and framing bias also feed the base-rate fallacy.
Base-rate neglect is particularly pronounced when there is a good, strong story.
Base-rate neglect is particularly pronounced when there is a good, strong story. Big, monumental projects typically have such a story, contributing to extra base-rate neglect for those.
Finally, people – including experts – underestimate variance. In the typical project, base-rate neglect therefore combines with variation neglect, in accordance with the following formula:
Base-rate neglect + variation neglect = strong convexity
Preliminary results from our research indicate that variation neglect receives less attention in decision making than base-rate neglect, which is unfortunate, because the research also indicates that variation neglect is typically larger and has even more drastic impact on project outcomes than base-rate neglect.
The base-rate fallacy runs rampant in project planning and management, as documented by the Iron Law of Project Management. Table 1 shows the most comprehensive overview that exists of base rates for costs and benefits in project management, based on data from 2,062 projects covering eight project types.
Most projects do not get base rates right – not even close.
We see that most projects do not get base rates right – not even close, as documented by averages that are different from one (1.0 ≈ correct base rate) at a level of statistical significance so high (p < 0.0001 ≈ "overwhelmingly high," in statisticians’ language) it is rarely found in studies of human behavior. The base-rate fallacy is deeply entrenched in project management, the data show. Flyvbjerg and Bester (2021) argue that base-rate neglect results in a new behavioral bias, which we call the "cost-benefit fallacy," which routinely derail cost-benefit analyses of projects to a degree where such analyses cannot be trusted.
As pointed out by Kahneman (2011: 150), "anyone who ignores base rates and the quality of evidence in probability assessments will certainly make mistakes." The cure to the base-rate fallacy, in and out of project management, is to get base rates right by taking an outside view, for instance through reference class forecasting, carrying out premortems, or doing decision hygiene (Flyvbjerg 2006, Klein 2007, Kahneman et al. 2011, 2021: 312–324, 371–72).
If you’re a project planner or manager, the easiest and most effective way to get started with curbing behavioral biases in your work is getting your base rates right, for the projects you’re responsible for.
If you don’t understand the real odds of a game, you’re unlikely to succeed at it.
Hopefully, most can see that if you don’t understand the real odds of a game, you’re unlikely to succeed at it. But that’s the situation for most project planners and managers: they don’t get the odds right for the game they are playing: project management. Table 1 documents this beyond reasonable doubt and establishes realistic base rates for a number of important areas in project management that planners can use as a starting point for getting their projects right. Data for other project types were not included for reasons of space but I have the data and they show similar results.
………………..
*) For full references, see Flyvbjerg, Bent, 2021, "Top Ten Behavioral Biases in Project Management: An Overview," Project Management Journal, vol. 52, no. 6, pp. 531–546. Free download here.
Correct answer to the quiz at the top of the page: 41%.
