SAM L. SAVAGE is a Consulting Professor of Management Science and Engineering at Stanford University, and a Fellow of the Judge Business School at the University of Cambridge.

• Foreword xvPreface xviiAcknowledgments xxiIntroduction Connecting the Seat of the Intellect to the Seat of the Pants 1You cannot learn to ride a bicycle from a book,and I claim the same is true for coping with uncertainty. Paradoxically, this book attempts to dowhat it claims is impossible.FoundationsPart 1 The Big Picture 9Chapter 1 The Flaw of Averages 11In planning for the future, uncertain outcomes are often replaced with single, so-called average numbers. This leads to a class of systematic errors that I call the Flaw of Averages, which explains among other things why forecasts are always wrong.Chapter 2 The Fall of the Algebraic Curtain and Rise of the Flaw of Averages 22The electronic spreadsheet brought the power of business modeling to tens of millions. In so doing, it also paved the way for an epidemic of the Flaw of Averages.Chapter 3 Mitigating the Flaw of Averages 26New technologies are illuminating uncertainty much as the lightbulb illuminates darkness. Probability Management is a scientific approach to harnessing these developments to cure the Flaw of Averages.Chapter 4 The Wright Brothers Versus the Wrong Brothers 34The success of the Wright Brothers’ airplane was the result of carefully constructed models that they tested in their wind tunnel. Analogous models ca help us manage uncertainty and risk, but as we saw in the financial crash of 2008, models can also be used to obfuscate.Chapter 5 The Most Important Instrument in the Cockpit 40The proper use of models, like the instruments in an airplane, is not obvious.Part 2 Five Basic Mindles for Uncertainty 45Chapter 6 Mindles are to Minds What Handles are to Hands 49Just as industrial designers develop handles to help us grasp the power of physics with our hands, informational designers develop Mindles (first syllable rhymes with “mind”) to help us grasp the power of information with our minds. Section 2will provide some important Mindles for grasping uncertainty.Chapter 7 Mindle 1: Uncertainty Versus Risk 52These two concepts are often used interchangeably but they shouldn’t be. Uncertainty is an objective feature of the universe, whereas risk is in the eye of the beholder.Chapter 8 Mindle 2: An Uncertain Number Is a Shape 55Even graduates of statistics courses have a hard time visualizing uncertainty. A shape in the form of a simple bar graph, called the histogram, does the trick. Try running a simulation in your head or better yet, on your web browser at FlawOfAverages.com.Chapter 9 Mindle 3: Combinations of Uncertain Numbers 67When uncertain numbers are added or averaged, the chance of extreme events goes down. I cover a case study in the film industry.Chapter 10 I Come to Bury Sigma, Not to Praise it 78Just as the height and weight of a criminal suspect have been superseded by surveillance videos and DNA samples, sigma is pushing obsolescence.Chapter 11 Mindle 4: Terri Dial and the Drunk in the Road 83A banking executive discovers the Strong Form of the Flaw of Averages: Average inputs don’t always result in average outputs. Designing an incentive plan around your average employee is systematically erroneous.Chapter 12 Who Was Jensen and Why Wasn’t He Equal? 91The Nuts and Bolts of the Strong Form of the Flaw of AveragesThis chapter shows how to identify the Flaw of Averages before it occurs by understanding your options and restrictions.Chapter 13 Mindle 5: Interrelated Uncertainties 98Interrelated uncertainties are at the heart of modern portfolio theory. They are best understood in terms of scatter plots.Part 3 Decisions and Information 109Chapter 14 Decision Trees 111Decision trees are a powerful Mindle for thinking through decisions in the face of uncertainty.Chapter 15 The Value of Information Because There Isn’t Anything Else 118The flip side of decision trees. Information is the complement of uncertainty. What is it worth to find things out?Part 4 The Seven Deadly Sins of Averaging 127Chapter 16 The Seven Deadly Sins of Averaging 129So here are the Seven Deadly Sins, all eleven of them. And the twelfth deadly sin is believing we won’t discover even more tomorrow.Chapter 17 The Flaw of Extremes 133Viewing uncertainties solely in terms of nonaverage outcomes also leads to devastatingly wrong answers and policy decisions.Chapter 18 Simpson’s Paradox 139Imagine a weight loss treatment that makes people lose weight on average, unless they are either male or female, in which case it makes them gain weight on average.Chapter 19 The Scholtes Revenue Fallacy 142Suppose you have various product lines with different unit sales. The average unit sales times the average profit per unit might be positive while your average profit might be negative.Chapter 20 Taking Credit for Chance Occurrences 147If you execute a marketing campaign and make a bunch of sales, how do you know the increase wasn’t just by chance?ApplicationsPart 5 The Flaw of Averages in Finance 155Chapter 21 Your Retirement Portfolio 157If your retirement fund will last you 20 years given average returns, then you are as likely as not to suffer financial ruin before you get there.Chapter 22 The Birth of Portfolio Theory: The Age of Covariance 163Harry Markowitz started a revolution in finance in the early 1950s by explicitly recognizing risk/return trade-offs.Chapter 23 When Harry Met Bill(y) 169Bill Sharpe extended the work of Markowitz and brought it into widespread practice.Chapter 24 Mindles for the Financial Planning Client 175How the pros explain this stuff to their clients.Chapter 25 Options: Profiting from Uncertainty 181Options allow us to exploit uncertainty through an understanding of the Strong Form of the Flaw of Averages.Chapter 26 When Fischer and Myron Met Bob: Option Theory 192The theory of three economists led to the trillion-dollar derivatives industry.Chapter 27 Prices, Probabilities, and Predictions 200The new phenomenon of prediction markets is changing the way we perceive and report uncertain events, such as political races.Part 6 Real Finance 213Chapter 28 Holistic Versus Hole-istic 215When people invest in portfolios of oil exploration sites, they often use the hole-istic approach. That is, they rank the places to drill hole by hole, then start at the top and go down the list until they run out of money. This ignores the holistic effects of portfolios.Chapter 29 Real Portfolios at Shell 222For several years, Shell has been using Probability Management to manage its portfolios of petroleum exploration sites in a more holistic manner.Chapter 30 Real Options 228An example of a real option is a gas well in which you have the choice of whether or not to pump depending on the price of gas.Chapter 31 Some Gratuitous Inflammatory Remarks on the Accounting Industry 236You can’t rely on accountants to detect risks because generally accepted accounting principles are built on the Flaw of Averages.Part 7 The Flaw of Averages in Supply Chains 245Chapter 32 The DNA of Supply Chains 247The inventory problem introduced in Chapter 1 is at the heart of all supply chains.Chapter 33 A Supply Chain of DNA 254When stocking out is not an option.Chapter 34 Cawlfield’s Principle 257A manager at Olin creates a simulation to get two divisions of his organization to work as a team and discovers a general principle in the process.Part 8 The Flaw of Averages and Some Hot Button Issues 263Chapter 35 The Statistical Research Group of World War II 265The exciting environment in which my father became a statistician.Chapter 36 Probability and the War on Terror 272Two inescapable statistical trademarks of the war on terror are the problem of false positives and implications of Markov chains.Chapter 37 The Flaw of Averages and Climate Change 289The earth’s average temperature may actually be going down, not up, but you won’t be happy when you find out why. The Flaw of Averages permeates this issue.Chapter 38 The Flaw of Averages in Health Care 299Treating the average patient is not healthy.Chapter 39 Sex and the Central Limit Theorem 307Women have a diversified portfolio of two X chromosomes, whereas men have only one. Apparently it makes a difference.Probability ManagementPart 9 Toward a Cure for the Flaw of Averages 317Chapter 40 The End of Statistics as You Were Taught It 319The ninetheenth-century statisticians confirmed their theories by simulating uncertainty with dice, cards, and numbered balls. Today, computerized dice, cards, and balls are bypassing the very theories they were trying to confirm.Chapter 41 Visualization 324Visual statistics provides a window into distributions. You need to see it to appreciate it.Chapter 42 Interactive Simulation: A New Lightbulb 328Imagine simulating 100,000 rolls of a die before your finger leaves the Enter key. A new technology does for probability distributions what the spreadsheet did for numbers.Chapter 43 Scenario Libraries: The Power Grid 332New data structures allow the results of simulations to be added together like numbers, providing a more practical approach to enterprisewide risk models.Chapter 44 The Fundamental Identity of SLURP Algebra 341This looks like math. Feel free to skip it.Chapter 45 Putting It into Practice 343The technology surrounding Probability Management is improving fast, and recent breakthroughs promises to make it more accessible than ever.Chapter 46 The CPO: Managing Probability Management 354The CPO must strike the correct balance between transparency of presentation, data collection, and statistical rigor.Chapter 47 A Posthumous Visit by My Father 364Some comments from the hereafter.Red Word Glossary 367Notes 371About the Author 382Index 383

Select Guide Rating