Why do studies show that 90% of worldwide transportation projects underestimate the actual costs that are incurred?1 It is not for lack of very detailed plans or resources. Research suggests that it may be due to holding a less than optimal viewpoint; however, improved decisions as a result of more accurate forecasts can be achieved with a simple framework. Herein we describe why this framework works and how you can apply it to achieve more success.
Biases in Decision Making
We all have many ingrained cognitive biases in our thinking that affect our decision-making. Researchers Daniel Kahneman and Amos Tversky found that people are generally overconfident when they make forecasts and that their estimates could be improved by examining the data in similar situations. The researchers call this the planning fallacy.2
Kahneman describes an example of when he fell prey to the planning fallacy. He was working on developing a curriculum and writing a textbook to teach judgment and decision-making in high school. His team included experienced teachers and the dean of a local university. After about a year of making some progress on the project, Kahneman asked each member of his team to estimate how long it would take them to finish the draft of the textbook. The responses ranged from one and a half to two and a half years with an average of two years.
Then Kahneman asked the dean to think about his extensive experience with other teams who had worked on creating textbooks from scratch and how long it had taken to develop from a similar point of progress. The dean thought carefully and finally replied that it had not occurred to him but many – about 40% – actually failed to complete the task from a similar point in their development. Those that did finish took an additional seven to ten years. Although Kahneman admits they should have quit right after this revelation because none of them wanted to invest six more years in a project with a 40% chance of failure, they still persisted. It took eight more years to finish the book!
The Inside View
Kahneman’s team’s overly optimistic forecast was a classic example of the inside view. Interestingly, they had carefully thought about specific details, such as how long it would take to complete each section. But some things are difficult to incorporate into a plan, no matter how detailed. How could Kahneman and his team know that some of them would fall ill and others would have personal problems that would cause them to stop working on the project for months at a time? When looking only at your own situation, these potential issues are very difficult to incorporate and almost impossible to quantify.
Better Forecasting with the Outside View
Problems and pitfalls that may have been unknowable do not have to be unpredictable. While they likely cannot be explicitly incorporated into a forecast, they are implicitly quantified and embedded within the outside view. That is because the outside view is based on “reference class forecasting,” or the incorporation of a wide array of data from comparable projects.
While the inside view is derived from a specific circumstance and uses evidence from one’s own experience, the outside view is derived from the reference class and uses evidence from others’ experiences.
| Derived from specific circumstances | Derived from the reference class |
| Uses evidence from own experiences | Uses evidence from others’ experiences |
| Qualitatively formed | Quantitatively grounded |
| Accounts for known variables | Can account for unknowns |
Planning the construction of a hypothetical house illustrates the steps involved in using reference classes and the outside view framework.
-
Identify a comparable reference class. In this example, the reference class is home construction.
-
Aggregate the data of the reference class to determine a baseline estimate (e.g., cost or time to complete construction).
-
Adjust the baseline estimate by the specific circumstance (e.g., rockier land or more experienced builders).
By incorporating the outcomes of many other homebuilders into the particular construction project in question, essentially unknowable issues are embedded in the estimate, making for what is likely to be a more accurate prediction.
Skill vs. Luck
The question of how much to adjust the baseline forecast is a tricky one but we can apply a framework to this problem: the more luck there is in the project or contest in question, the less adjustment should be made based on specific circumstances. Conversely, the more skill that is present, the more specific adjustments should be made to the baseline.
For example, suppose Stephen Curry, the sharp-shooting point guard for the Golden State Warriors, is tossing a coin and has been relatively successful in trying to get heads to come up, with 57% of his past 100 flips being heads. What is the next toss likely to be? Because the game is pure luck, the reference class forecast would give you the right answer. Namely, the average toss of a coin results in a 50-50 probability of heads-tails so that is the probability for the next toss. When determining the probability of a flip being heads there is no need to incorporate Curry’s skills.
It is another matter to forecast the outcome of foul shots, which involve a high level of skill. If the reference class is the average successful free-throw percentage in the NBA, which is in the mid-70% range, then the probability of Curry hitting foul shots should be adjusted significantly higher. That is because the outcome of a free throw shot is materially impacted by ability and Curry is really good at it – he has averaged a stunning 90% over his career.
Application to Investing
The inside vs. outside view can be applied to investing as well. We frequently see the inside view in the media. TV pundits often discuss future equity returns in terms of economic indicators, earnings reports, and central bank actions. These issues are all important but they most often focus on the specifics of the current situation, which is the basis for the inside view. By contrast, the outside view would begin by looking at a set of statistics for a comparable situation or reference class. For example, when looking back at the past several decades, we find that over 80% of the variation in 10-year annual equity returns is determined by the aggregate stock market price-to-earnings multiple (P/E). Assuming this historical relationship continues, one could estimate a projection for a 10-year annualized equity return with reasonable confidence. This estimated return could then potentially be adjusted somewhat for current circumstances.
This type of analysis can also be used when forecasting company-specific fundamentals and stock prices. By having experience with a multitude of companies, analysts can make judgments about, say, how long a given company’s hyper-growth may last in the context of other companies or industries over history. Making these forecasts in the context of a rich data set is likely to be more accurate than simply listening to management and judging a company’s prospects in isolation.
In investing and in life, the outside view is more likely to result in more accurate forecasts and better outcomes for the decision-maker. We should all spend more time looking outside of our particular situations to make better decisions.
