The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty

Table of Contents:
Foreword xv Preface xvii Acknowledgments xxi Introduction Connecting the Seat of the Intellect to the Seat of the Pants 1 You cannot learn to ride a bicycle from a book,and I claim the same is true for coping with uncertainty. Paradoxically, this book attempts to dowhat it claims is impossible. Foundations Part 1 The Big Picture 9 Chapter 1 The Flaw of Averages 11 In planning for the future, uncertain outcomes are often replaced with single, so-called average numbers. This leads to a class of systematic errors that I call the Flaw of Averages, which explains among other things why forecasts are always wrong. Chapter 2 The Fall of the Algebraic Curtain and Rise of the Flaw of Averages 22 The electronic spreadsheet brought the power of business modeling to tens of millions. In so doing, it also paved the way for an epidemic of the Flaw of Averages. Chapter 3 Mitigating the Flaw of Averages 26 New technologies are illuminating uncertainty much as the lightbulb illuminates darkness. Probability Management is a scientific approach to harnessing these developments to cure the Flaw of Averages. Chapter 4 The Wright Brothers Versus the Wrong Brothers 34 The success of the Wright Brothers’ airplane was the result of carefully constructed models that they tested in their wind tunnel. Analogous models ca help us manage uncertainty and risk, but as we saw in the financial crash of 2008, models can also be used to obfuscate. Chapter 5 The Most Important Instrument in the Cockpit 40 The proper use of models, like the instruments in an airplane, is not obvious. Part 2 Five Basic Mindles for Uncertainty 45 Chapter 6 Mindles are to Minds What Handles are to Hands 49 Just as industrial designers develop handles to help us grasp the power of physics with our hands, informational designers develop Mindles (first syllable rhymes with “mind”) to help us grasp the power of information with our minds. Section 2will provide some important Mindles for grasping uncertainty. Chapter 7 Mindle 1: Uncertainty Versus Risk 52 These two concepts are often used interchangeably but they shouldn’t be. Uncertainty is an objective feature of the universe, whereas risk is in the eye of the beholder. Chapter 8 Mindle 2: An Uncertain Number Is a Shape 55 Even graduates of statistics courses have a hard time visualizing uncertainty. A shape in the form of a simple bar graph, called the histogram, does the trick. Try running a simulation in your head or better yet, on your web browser at FlawOfAverages.com. Chapter 9 Mindle 3: Combinations of Uncertain Numbers 67 When uncertain numbers are added or averaged, the chance of extreme events goes down. I cover a case study in the film industry. Chapter 10 I Come to Bury Sigma, Not to Praise it 78 Just as the height and weight of a criminal suspect have been superseded by surveillance videos and DNA samples, sigma is pushing obsolescence. Chapter 11 Mindle 4: Terri Dial and the Drunk in the Road 83 A banking executive discovers the Strong Form of the Flaw of Averages: Average inputs don’t always result in average outputs. Designing an incentive plan around your average employee is systematically erroneous. Chapter 12 Who Was Jensen and Why Wasn’t He Equal? 91 The Nuts and Bolts of the Strong Form of the Flaw of Averages This chapter shows how to identify the Flaw of Averages before it occurs by understanding your options and restrictions. Chapter 13 Mindle 5: Interrelated Uncertainties 98 Interrelated uncertainties are at the heart of modern portfolio theory. They are best understood in terms of scatter plots. Part 3 Decisions and Information 109 Chapter 14 Decision Trees 111 Decision trees are a powerful Mindle for thinking through decisions in the face of uncertainty. Chapter 15 The Value of Information Because There Isn’t Anything Else 118 The flip side of decision trees. Information is the complement of uncertainty. What is it worth to find things out? Part 4 The Seven Deadly Sins of Averaging 127 Chapter 16 The Seven Deadly Sins of Averaging 129 So here are the Seven Deadly Sins, all eleven of them. And the twelfth deadly sin is believing we won’t discover even more tomorrow. Chapter 17 The Flaw of Extremes 133 Viewing uncertainties solely in terms of nonaverage outcomes also leads to devastatingly wrong answers and policy decisions. Chapter 18 Simpson’s Paradox 139 Imagine a weight loss treatment that makes people lose weight on average, unless they are either male or female, in which case it makes them gain weight on average. Chapter 19 The Scholtes Revenue Fallacy 142 Suppose you have various product lines with different unit sales. The average unit sales times the average profit per unit might be positive while your average profit might be negative. Chapter 20 Taking Credit for Chance Occurrences 147 If you execute a marketing campaign and make a bunch of sales, how do you know the increase wasn’t just by chance? Applications Part 5 The Flaw of Averages in Finance 155 Chapter 21 Your Retirement Portfolio 157 If your retirement fund will last you 20 years given average returns, then you are as likely as not to suffer financial ruin before you get there. Chapter 22 The Birth of Portfolio Theory: The Age of Covariance 163 Harry Markowitz started a revolution in finance in the early 1950s by explicitly recognizing risk/return trade-offs. Chapter 23 When Harry Met Bill(y) 169 Bill Sharpe extended the work of Markowitz and brought it into widespread practice. Chapter 24 Mindles for the Financial Planning Client 175 How the pros explain this stuff to their clients. Chapter 25 Options: Profiting from Uncertainty 181 Options allow us to exploit uncertainty through an understanding of the Strong Form of the Flaw of Averages. Chapter 26 When Fischer and Myron Met Bob: Option Theory 192 The theory of three economists led to the trillion-dollar derivatives industry. Chapter 27 Prices, Probabilities, and Predictions 200 The new phenomenon of prediction markets is changing the way we perceive and report uncertain events, such as political races. Part 6 Real Finance 213 Chapter 28 Holistic Versus Hole-istic 215 When people invest in portfolios of oil exploration sites, they often use the hole-istic approach. That is, they rank the places to drill hole by hole, then start at the top and go down the list until they run out of money. This ignores the holistic effects of portfolios. Chapter 29 Real Portfolios at Shell 222 For several years, Shell has been using Probability Management to manage its portfolios of petroleum exploration sites in a more holistic manner. Chapter 30 Real Options 228 An example of a real option is a gas well in which you have the choice of whether or not to pump depending on the price of gas. Chapter 31 Some Gratuitous Inflammatory Remarks on the Accounting Industry 236 You can’t rely on accountants to detect risks because generally accepted accounting principles are built on the Flaw of Averages. Part 7 The Flaw of Averages in Supply Chains 245 Chapter 32 The DNA of Supply Chains 247 The inventory problem introduced in Chapter 1 is at the heart of all supply chains. Chapter 33 A Supply Chain of DNA 254 When stocking out is not an option. Chapter 34 Cawlfield’s Principle 257 A manager at Olin creates a simulation to get two divisions of his organization to work as a team and discovers a general principle in the process. Part 8 The Flaw of Averages and Some Hot Button Issues 263 Chapter 35 The Statistical Research Group of World War II 265 The exciting environment in which my father became a statistician. Chapter 36 Probability and the War on Terror 272 Two inescapable statistical trademarks of the war on terror are the problem of false positives and implications of Markov chains. Chapter 37 The Flaw of Averages and Climate Change 289 The earth’s average temperature may actually be going down, not up, but you won’t be happy when you find out why. The Flaw of Averages permeates this issue. Chapter 38 The Flaw of Averages in Health Care 299 Treating the average patient is not healthy. Chapter 39 Sex and the Central Limit Theorem 307 Women have a diversified portfolio of two X chromosomes, whereas men have only one. Apparently it makes a difference. Probability Management Part 9 Toward a Cure for the Flaw of Averages 317 Chapter 40 The End of Statistics as You Were Taught It 319 The ninetheenth-century statisticians confirmed their theories by simulating uncertainty with dice, cards, and numbered balls. Today, computerized dice, cards, and balls are bypassing the very theories they were trying to confirm. Chapter 41 Visualization 324 Visual statistics provides a window into distributions. You need to see it to appreciate it. Chapter 42 Interactive Simulation: A New Lightbulb 328 Imagine simulating 100,000 rolls of a die before your finger leaves the Enter key. A new technology does for probability distributions what the spreadsheet did for numbers. Chapter 43 Scenario Libraries: The Power Grid 332 New data structures allow the results of simulations to be added together like numbers, providing a more practical approach to enterprisewide risk models. Chapter 44 The Fundamental Identity of SLURP Algebra 341 This looks like math. Feel free to skip it. Chapter 45 Putting It into Practice 343 The technology surrounding Probability Management is improving fast, and recent breakthroughs promises to make it more accessible than ever. Chapter 46 The CPO: Managing Probability Management 354 The CPO must strike the correct balance between transparency of presentation, data collection, and statistical rigor. Chapter 47 A Posthumous Visit by My Father 364 Some comments from the hereafter. Red Word Glossary 367 Notes 371 About the Author 382 Index 383

About the Author :
SAM L. SAVAGE is a Consulting Professor of Management Science and Engineering at Stanford University, and a Fellow of the Judge Business School at the University of Cambridge.