Review
How to Measure Anything isn't a product book, and I'd still put it on the required list for product managers. Discovery is fundamentally about reducing uncertainty before you commit resources, and Hubbard's entire thesis is a rigorous, practical answer to how you actually do that. His core reframe stuck with me: measurement isn't about eliminating uncertainty, it's about reducing it, and even a small amount of early observation reduces it more than people expect.
What makes the book genuinely useful rather than just interesting is that Hubbard shows you how to quantify things that feel unmeasurable: user preferences, market demand, the impact of a feature nobody's shipped yet. None of it requires perfect data. Rough estimates, done with the right method, get you most of the way there.
It's not a quick read, and some of it is drier statistics than most PMs will want. Worth the investment anyway.
Key Takeaways
The parts worth keeping:
The core idea
- If something matters at all, it has observable consequences, which means it can be measured. "Immeasurable" is almost always a failure of imagination, not a fact about the thing itself.
- Measurement is a quantitatively expressed reduction of uncertainty based on observation, not the elimination of it. That distinction is the whole book.
- The five-step framework: define the decision, determine what you already know, compute the value of additional information (skip to step 5 if there isn't any), measure where that value is high, then decide and act, which creates the next decision to repeat the loop on.
You need less data than you think
- The Rule of Five: take any five random samples from a population, and there's a 93.75% chance the true median falls between the smallest and largest of them. Wide, but far better than nothing.
- The information value curve is steepest at the start. Your first ten samples tell you far more than your next ten, because your initial state of uncertainty was so high.
- Decomposing an uncertain variable into smaller, more measurable parts often reduces uncertainty enough on its own, no further measurement needed for roughly a third of the variables people decompose this way.
Finding data you didn't know you had
Most things people call hard to measure just require indirect inference rather than direct observation, the way Eratosthenes measured the Earth's curvature from shadows without ever seeing the curve itself. Hubbard's approach: follow the trail. Do forensic analysis on data you already have, start directly observing or sampling if you can, add a tracer if nothing currently leaves a trail, and if none of that works, create an experiment that forces one into existence. Controlled experiments (test group against control) are what get you from correlation to causation. And remember that absence of evidence is, in fact, evidence of absence, despite the cliche claiming otherwise.
Where the effort actually pays off
- Expected Opportunity Loss (EOL) is the chance you're wrong multiplied by the cost of being wrong. The Expected Value of Information is how much a measurement would reduce that EOL, and it's what should decide where you spend measurement effort, not habit or convenience.
- Measurement Inversion: the variables organizations measure most are usually the ones with the least information value, because people default to measuring what's easy rather than what's uncertain and costly to get wrong.
- The Risk Paradox follows from this: quantitative risk analysis mostly gets applied to routine, low-stakes decisions. The biggest, riskiest calls get the least rigorous analysis, exactly backwards from where it would help most.
- Managers gravitate toward measuring things that produce good news. Worth watching for in yourself.
Getting better at estimating
Calibration is a trainable skill, not a fixed trait. A few techniques that help: imagine an equivalent bet with a 90% payoff and adjust your confidence interval until you're indifferent between the two; treat each bound of a range as its own yes/no question rather than anchoring on a single point estimate; actively look for reasons your estimate could be wrong instead of just defending it; and when anchoring is dragging your range too narrow, start absurdly wide and eliminate the values you know are ridiculous, rather than starting narrow and expanding.
One common error to avoid: building a "pessimistic case" from all-pessimistic inputs and an "optimistic case" from all-optimistic ones. The more variables involved, the more that combination exaggerates the true range, and the worse the resulting decision.