Strong Inference

John R. Platt · 1964 · Science 146(3642), 347-353

The argument

Platt opens with a simple observation. Molecular biology and certain branches of physics advance rapidly. Whole fields of knowledge open up in a decade. Other sciences, equally well-funded and equally staffed with talented people, seem to wander. What explains the difference?

His answer: the fast-moving fields practice a systematic method he calls strong inference. Three steps, applied to every problem. First, devise alternative hypotheses. Not one hypothesis and a null. Multiple, mutually exclusive explanations for the same phenomenon. Second, design a crucial experiment with alternative possible outcomes, each of which excludes one or more hypotheses. Third, carry out the experiment to get a clean result. Then recycle: the surviving hypotheses generate new alternatives, new experiments, new eliminations.

The method is not new. Platt credits 🔬 Chamberlin (1890) for the idea of multiple working hypotheses. What Platt adds is the insistence that the method be applied systematically, at every step, as institutional habit rather than occasional insight.

Strong vs. weak inference

Weak inference gathers data and hopes patterns emerge. It measures everything available, runs correlations, reports what passes a significance threshold. There is no branching logic, no elimination, no tree of decreasing possibilities. The researcher accumulates findings. Progress is additive.

Strong inference is subtractive. Each experiment kills something. The surviving hypothesis space shrinks. Molecular biologists in the 1950s and 1960s moved fast because every experiment was designed to tell them which model was wrong. They could answer "is the genetic code overlapping or non-overlapping?" with a single experiment. They did not collect data about the genetic code in general and hope to figure it out later.

Platt is blunt about which fields practice which. Physics and molecular biology practice strong inference. Much of psychology, much of ecology, much of social science practices weak inference. The difference in rate of progress is not a coincidence.

What it requires

Holding multiple hypotheses simultaneously is uncomfortable. The natural tendency is to commit to one explanation and look for evidence that supports it. Chamberlin described this failure mode in 1890: the "ruling theory" that becomes a "ruling passion." Platt operationalizes the fix. You write down all the plausible alternatives before you design the experiment. You design the experiment so that the outcome distinguishes between them. If your experiment is compatible with all your hypotheses regardless of outcome, it is not a crucial experiment. Redesign it.

This discipline has a cost. It forces you to take seriously explanations you find implausible. It forces you to invest effort designing experiments that might prove you wrong. It forces clarity about what you would accept as evidence against your preferred explanation. Most researchers find this unpleasant. The fields that do it anyway are the ones that move.

Discussion

Platt operationalized what Chamberlin described. Chamberlin said: hold multiple hypotheses. Platt said: here is the protocol. Devise alternatives, design crucial experiments, carry them out, repeat. The contribution is not philosophical. It is procedural.

🔬 Popper argued that science advances by falsification. Strong inference is applied falsification with a twist: you falsify not against a single hypothesis but against a branching tree of alternatives. Each experiment prunes branches. Popper gave the logic. Platt gave the workflow.

Feynman arrives at the same place from a different direction. Feynman does not talk about protocol. He talks about integrity: the willingness to design experiments that could prove you wrong, to report what went wrong, to mention alternative explanations. Strong inference is what Feynman's integrity looks like as a daily practice. You cannot fool yourself about which hypothesis survives if you designed the experiment to eliminate the losers cleanly.

The integrity angle

Confirmation bias is the default. Without deliberate structure, researchers gravitate toward experiments that can confirm but not disconfirm their favored hypothesis. Platt's protocol is a structural defense against this tendency. It works not by making scientists more honest but by making dishonesty harder to sustain. When you have three hypotheses and an experiment eliminates two, the result is public and unambiguous. There is no room for reinterpretation.

The discipline of designing experiments that discriminate rather than confirm is rare because it requires accepting that most of your ideas are wrong. Each round of strong inference kills more hypotheses than it spares. Researchers who practice it spend most of their time being wrong. Researchers who practice weak inference can avoid that experience indefinitely. The difference shows up in the rate of actual progress, not in the number of publications.