Review: The Flaw of Averages, by Sam L. Savage
By David H. Mirman
The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty, by Sam L. Savage. Hoboken, NJ: John Wiley & Sons Inc., 2009. 412 pp. $27.95 (cloth).
Consider a drunk staggering down the middle of a busy highway and assume that his average position is the centerline. The state of the drunk at his average position is alive, but on average, he’s dead. (p. 83)
This). “Plans based on average assumptions are wrong on average,” he explains. “In everyday life, the Flaw of Averages ensures that plans based on the average customer demand, average completion time . . . and other uncertainties are . . . behind schedule, and beyond budget” (p. 11, emphasis original). In The Flaw of Averages, Savage demonstrates the harm caused by the error and arms his readers with skills to avoid it.
Toward helping the reader understand the concepts of probability and statistics needed to grasp the Flaw, Savage uses a radically different approach than is typically used in college statistics courses. He begins by describing the motion of a bicycle with a series of differential equations, thereby leading the reader to believe that such mathematical formalities are in store for him throughout the book.
Just kidding! . . . You have solved [these equations] for most of your life, not through the seat of your intellect, but experientially, through the seat of your pants. The theory of probability and statistics can likewise be presented in terms of mind-numbing equations, and that’s the way it’s usually taught in school. (p. 2)
Savage dispenses with the mind-numbing equations in favor of graphs and whimsical illustrations (many by syndicated cartoonist Jeff Danziger). Savage also provides, via a free companion website (www.flawofaverages.com), animations and simulations that allow the reader to generate data dynamically. In one simulation, for example, the viewer clicks on two spinners to generate random numbers and observes a histogram of their sum gradually take on the infamous “bell curve” shape. Further, Savage simplifies the discussion through his use of plain English or what he calls “Green Words” rather than technical terms or “Red Words” (p. 45–46). He asks the reader to think in simple terms such as “interrelationships,” “degree of uncertainty,” and “upward and downward curves” rather than “covariance,” “variance/standard deviation/sigma,” and “convex and concave functions.” Savage also concretizes his points with an engaging mixture of simplified hypothetical examples and real-world applications from his work as a consultant. In one case, for instance, he employs details from his work with Shell Oil, in which he showed the company how to determine which combinations of exploration projects would be most profitable. Through these and other means, Savage succeeds in making complex statistical concepts accessible to the general reader.
Having established his general approach, Savage shows the Flaw of Averages in action. In one case, he shows how it can lead to the unrealistic scheduling of a large software project involving ten programmers. A manager asks one programmer how long his portion of the project will take, and the programmer replies, “Three to nine months, six months on average.” As the actual time needed is uncertain, the manager might reasonably estimate that this programmer’s portion of the project will take six months. If each of the ten programmers also expects to take six months on average, the manager might conclude that the entire software project will most likely be ready in about six months. Although this conclusion may appear valid, Savage shows that it is not. Each programmer has a possibility of finishing his segment earlier than six months and a possibility of finishing his segment later than six months. And, importantly, if any of them is late, he will delay the entire project. If it is equally likely for a programmer to fall behind as it is for him to finish early, then for each programmer there is a 50/50 chance of finishing early or late, just as a coin toss has a 50/50 chance of landing heads or tails. For all ten to finish in six months (i.e., for none to finish late) is analogous to flipping heads on a coin ten times in a row—the odds of which are less than one in one thousand (p. 14). So the manager who predicts that the software project will be done in six months will almost surely be wrong; statistically, there is a 99.9 percent chance of it. And his erroneous prediction is a classic example of the Flaw of Averages: By using the average of an uncertain number (the average time to complete a section of software) to represent a range of possible outcome (the full range of possible times each section might need), he is misled.
After discussing the basic form of the Flaw, Savage introduces a more harmful version, which he calls the “Strong Form of the Flaw of Averages.” This, too, he demonstrates with an example. Savage asks the reader to determine the value of a natural gas reservoir containing one million units of natural gas, assuming that the cost to pump and sell a unit of gas is $9.50 and the average price of gas is $10 per unit. The reader will likely judge the reservoir to be worth $500,000—one million units times $0.50 profit per unit. But Savage explains that the reservoir is actually worth more—much more:
If the gas price goes up, the value goes up. “But what if the price goes down?” you say. “You could lose a lot of money in that case.” No you couldn’t. If gas drops below $9.50, you won’t pump. You have the option not to pump, and this limits the downside to zero. (p. 88)
He illustrates by considering a case in which the gas price is expected to fluctuate between $2.50 above and $2.50 below the $10 average. The profit would range from $0 (when the gas price is $7.50 and you decide not to pump) to $3 million (when the gas price is $12.50 and you make $3.00 [$12.50 minus $9.50] per unit times one million units). “The average of $3 million and zero is $1.5 million, three times the profit associated with the average gas price!” (p. 89). This demonstrates the Strong Form of the Flaw, which Savage defines as “the average or expected inputs don’t always result in average or expected output” (p. 90). This is shown in the above example: The average price of gas (the input) does not accurately predict the expected value of the reservoir (output). Savage concludes: “Now what you choose to pay for such a property is your business, but if you do not recognize that its average value is way over the value based on average gas prices, then you will be outbid by people who recognize this . . . opportunity” (p. 89).
To further help the reader understand how to properly think about and deal with uncertainty, Savage develops five “Mindles.” “Just as a handle helps you grasp something with your hand, I define a Mindle (first syllable rhymes with ‘mind’) as the analogous concept for the mind” (p. 45). For example, Mindle one, “Risk is in the eye of the beholder” (p. 54), distinguishes between risk and uncertainty, that is, between the “possibility of loss or injury” and “the quality or state of being uncertain: doubt” (p. 53). Savage illustrates the difference by comparing two individuals invested in the same stock: one owns shares of the stock and the other has a short position1 in that stock. Each investor is uncertain about the future share price; neither knows whether it will appreciate or decline. Their risks, however, are different: The investor who owns the shares faces the risk that the share price will decline, whereas the investor short on the stock faces the risk that the price will increase (pp. 53–54). Savage uses other “Mindles” to introduce the Strong Form of the Flaw of Averages to demonstrate how effectively histograms can represent the full range of possible outcomes of an uncertain number, to illustrate how diversification can reduce risk in investment portfolios, and to show that uncertainties can be interrelated.
The Foundations section of the book, which contains Savage’s exposition of the Flaw of Averages and the “Mindles,” is likely to contain most of the book’s value for the general reader. The subsequent Applications section includes topics in finance, supply chain management, the War on Terror, global climate change, health care, and sex. (Savage is personally acquainted with many of the individuals who developed portfolio theory, and those interested in finance will appreciate his insider’s perspective.) The Applications section suffers, though, from uneven coverage; the material on finance, for instance, is overly detailed whereas the coverage of health care, a field rife with the Flaw, is somewhat superficial.
In the third and final section of the book, Probability Management, Savage discusses the virtue of using specialized software to help avoid the Flaw of Averages. Although spreadsheet software was initially instrumental in proliferating the Flaw by “tempting millions of managers to plug in single numbers to represent uncertainties” (pp. 24–25), the modern software touted (and, in some cases, developed) by Savage is more sophisticated. Some of these are stand-alone programs whereas others are plug-ins that enable Microsoft Excel to handle a full distribution of values in one field as if it were one number (rather than replacing that distribution with its average) using a standardized format Savage developed called a “Distribution String” or “DIST” (p. 344). Those who deal with business models or other prediction methods involving uncertainty may appreciate the discussion of this software; however, those lacking need for such software may find little of interest here.
Although some of its material may exceed the general reader’s needs, The Flaw of Averages will prove valuable for anyone seeking clear guidance in accurately identifying and effectively mitigating risk in the face of uncertainty.