Cargo Cult Science

Richard P. Feynman · 1974 · Caltech Commencement Address · Caltech transcript

The argument

Feynman opens with the South Seas islanders. During World War II, cargo planes landed on their islands bringing supplies. After the war, the planes stopped. The islanders built wooden runways, lit signal fires, carved coconut-shell headphones, sat in bamboo control towers, and waited. They had the form exactly right. They were missing the substance.

Science can fail the same way. A researcher wears the lab coat, follows the protocol, runs the statistics, publishes in the journal. Everything looks correct. But the core honesty is missing. The experiments are not designed to catch the researcher being wrong. The negative results go unreported. The alternative explanations go unmentioned. The form of science is present. The substance is not.

Feynman does not blame fraud. Fraud is rare and obvious. The problem is subtler. Researchers fool themselves without knowing it. They design experiments that confirm rather than discriminate. They remember the successes and forget the failures. They interpret ambiguous results in favor of their hypothesis. None of this requires dishonesty in the ordinary sense. It requires only the absence of a specific, deliberate, uncomfortable practice: trying to prove yourself wrong.

What Feynman says integrity requires

Each of these is individually obvious. Together they constitute a practice that most published research does not follow. Feynman knew this in 1974. Ioannidis quantified it in 2005.

The examples

Feynman gives three cases. Each one shows the same failure: the form of science without the substance of self-correction.

Millikan measured the charge of the electron using oil drops. His published value was slightly wrong because of an incorrect value for the viscosity of air. Later experimenters who got values closer to the modern value but far from Millikan's threw out their results, assuming they had made an error. They crept toward the correct value slowly, always anchored to Millikan's number. Nobody published a dramatic correction. The field inched its way to the truth, discarding good data along the way because it did not match the authority.

Rat-running experiments in psychology showed a persistent pattern. Each experimenter who ran rats through mazes discovered specific confounds: the rats could smell the food, hear the experimenter's breathing, feel the floor texture. Each time, the confound was documented, and the next experimenter ignored the documentation and repeated the same mistakes. No cumulative learning. No institutional memory of what goes wrong.

Educational research tested new teaching methods by giving tests before and after. But the tests were not controlled for what students would have learned anyway. No comparison group. No blinding. No attempt to distinguish "this method works" from "students learn over time regardless." The form of experiment was present. The content was absent.

Discussion

🔬 Chamberlin anticipated the core problem. Holding multiple hypotheses prevents the "ruling theory" from becoming a "ruling passion." Feynman's integrity requirements are what multiple working hypotheses look like in practice. You cannot list alternative explanations if you have not entertained them. You cannot mention what might be wrong if you have only looked for what confirms your expectation.

🔬 Ioannidis (2005) showed that the cargo cult was not anecdotal. Most published research findings in medicine are false. The reasons are structural: small samples, small effects, large numbers of tested hypotheses, flexible designs, financial incentives, and the absence of replication. Everything Feynman warned about, measured at scale.

🔬 Mayo built the statistical framework for catching self-deception. Her severe testing criterion says: a test provides evidence for a claim only if the test had a high probability of detecting the claim's falsity, were it false. This is Feynman's integrity principle translated into statistical theory. A test that cannot fail is not a test. An experiment that confirms regardless of outcome is cargo cult science with a p-value attached.

Platt's strong inference is the daily practice that makes Feynman's integrity operational. Devise alternatives, design crucial experiments, eliminate hypotheses. Each round of elimination is an act of self-honesty. You designed the experiment to kill your idea if it deserves killing. That is the difference between a runway that lands planes and a runway made of bamboo.

Why this is the integrity centerpiece

Every other work in this collection describes a method, a logic, a protocol. Feynman describes a character trait. The methods can be followed mechanically. Integrity cannot. You can run a randomized controlled trial and still fool yourself about what it means. You can practice strong inference and still suppress the alternative you find threatening. The first principle is about you, not your protocol.

That is what makes it harder than the others. Chamberlin gives you a checklist: write down multiple hypotheses. Platt gives you a workflow: design experiments that discriminate. Popper gives you a criterion: a theory must be falsifiable. Feynman gives you a mirror. You must not fool yourself. The rest is commentary.

The lecture is four thousand words. It was delivered to graduating seniors who were about to enter careers in science and engineering. Feynman could have talked about the excitement of discovery or the importance of curiosity. He chose to talk about honesty. Not honesty toward others. Honesty toward yourself. That choice says everything about what he thought science needed most.