How to Measure Anything

DOUGLAS W. HUBBARD is the inventor of Applied Information Economics (AIE), a measurement methodology that has earned him critical praise from The Gartner Group, Giga Information Group, and Forrester Research. He is an internationally recognized expert in the field of decision analysis and challenging measurements and is a popular speaker at numerous conferences. He has written articles for Information Week, CIO Enterprise, and DBMS Magazine. He is the author of The Failure of Risk Management: Why It’s Broken and How to Fix It and Pulse: The New Science of Harnessing Internet Buzz to Track Threats and Opportunities.

• Preface to the Third Edition xiiiAcknowledgments xixAbout the Author xxiPart I The Measurement Solution Exists 1Chapter 1 The Challenge of Intangibles 3The Alleged Intangibles 4Yes, I Mean Anything 5The Proposal: It’s about Decisions 7A “Power Tools” Approach to Measurement 10A Guide to the Rest of the Book 11Chapter 2 An Intuitive Measurement Habit: Eratosthenes, Enrico, and Emily 15How an Ancient Greek Measured the Size of Earth 16Estimating: Be Like Fermi 17Experiments: Not Just for Adults 20Notes on What to Learn from Eratosthenes, Enrico, and Emily 25Notes 27Chapter 3 The Illusion of Intangibles: Why Immeasurables Aren’t 29The Concept of Measurement 30The Object of Measurement 37The Methods of Measurement 40Economic Objections to Measurement 48The Broader Objection to the Usefulness of “Statistics” 52Ethical Objections to Measurement 55Reversing Old Assumptions 58Notes 65Part II Before You Measure 69Chapter 4 Clarifying the Measurement Problem 71Toward a Universal Approach to Measurement 73The Unexpected Challenge of Defining a Decision 74If You Understand It, You Can Model It 80Getting the Language Right: What “Uncertainty” and “Risk” Really Mean 83An Example of a Clarified Decision 84Notes 90Chapter 5 Calibrated Estimates: How Much Do You Know Now? 93Calibration Exercise 95Calibration Trick: Bet Money (or Even Just Pretend To) 101Further Improvements on Calibration 104Conceptual Obstacles to Calibration 106The Effects of Calibration Training 111Notes 118Chapter 6 Quantifying Risk through Modeling 123How Not to Quantify Risk 123Real Risk Analysis: The Monte Carlo 125An Example of the Monte Carlo Method and Risk 127Tools and Other Resources for Monte Carlo Simulations 136The Risk Paradox and the Need for Better Risk Analysis 140Notes 143Chapter 7 Quantifying the Value of Information 145The Chance of Being Wrong and the Cost of Being Wrong: Expected Opportunity Loss 146The Value of Information for Ranges 149Beyond Yes/No: Decisions on a Continuum 156The Imperfect World: The Value of Partial Uncertainty Reduction 159The Epiphany Equation: How the Value of Information Changes Everything 166Summarizing Uncertainty, Risk, and Information Value: The Pre-Measurements 171Notes 172Part III Measurement Methods 173Chapter 8 The Transition: From What to Measure to How to Measure 175Tools of Observation: Introduction to the Instrument of Measurement 177Decomposition 180Secondary Research: Assuming You Weren’t the First to Measure It 184The Basic Methods of Observation: If One Doesn’t Work, Try the Next 186Measure Just Enough 188Consider the Error 189Choose and Design the Instrument 194Notes 196Chapter 9 Sampling Reality: How Observing Some Things Tells Us about All Things 197Building an Intuition for Random Sampling: The Jelly Bean Example 199A Little about Little Samples: A Beer Brewer’s Approach 200Are Small Samples Really “Statistically Significant”? 204When Outliers Matter Most 208The Easiest Sample Statistics Ever 210A Biased Sample of Sampling Methods 214Experiment 226Seeing Relationships in the Data: An Introduction to Regression Modeling 235Notes 243Chapter 10 Bayes: Adding to What You Know Now 247The Basics and Bayes 248Using Your Natural Bayesian Instinct 257Heterogeneous Benchmarking: A “Brand Damage” Application 263Bayesian Inversion for Ranges: An Overview 267The Lessons of Bayes 276Notes 282Part IV Beyond the Basics 285Chapter 11 Preference and Attitudes: The Softer Side of Measurement 287Observing Opinions, Values, and the Pursuit of Happiness 287A Willingness to Pay: Measuring Value via Trade-Offs 292Putting It All on the Line: Quantifying Risk Tolerance 296Quantifying Subjective Trade-Offs: Dealing with Multiple Conflicting Preferences 299Keeping the Big Picture in Mind: Profit Maximization versus Purely Subjective Trade-Offs 302Notes 304Chapter 12 The Ultimate Measurement Instrument: Human Judges 307Homo Absurdus: The Weird Reasons behind Our Decisions 308Getting Organized: A Performance Evaluation Example 313Surprisingly Simple Linear Models 315How to Standardize Any Evaluation: Rasch Models 316Removing Human Inconsistency: The Lens Model 320Panacea or Placebo?: Questionable Methods of Measurement 325Comparing the Methods 333Example: A Scientist Measures the Performance of a Decision Model 335Notes 336Chapter 13 New Measurement Instruments for Management 339The Twenty-First-Century Tracker: Keeping Tabs with Technology 339Measuring the World: The Internet as an Instrument 342Prediction Markets: A Dynamic Aggregation of Opinions 346Notes 353Chapter 14 A Universal Measurement Method: Applied Information Economics 357Bringing the Pieces Together 358Case: The Value of the System That Monitors Your Drinking Water 362Case: Forecasting Fuel for the Marine Corps 367Case: Measuring the Value of ACORD Standards 373Ideas for Getting Started: A Few Final Examples 378Summarizing the Philosophy 384Notes 385Appendix Calibration Tests (and Their Answers) 387Index 397

Select Guide Rating