Can You Trust a Famous Psychology Study?
From the Fifty Classic Studies in Psychology collection
In 1968, students sat alone in booths and heard a voice over an intercom begin to choke and call for help. When they believed nobody else could hear, 85% went for help before the seizure ended. When they believed four others were listening, only 31% did. The result is usually retold as a fact about human nature: crowds do not help.
A year later, researchers staged a collapse on New York subway trains. A man with a cane fell to the floor and was helped in 62 of 65 trials, often within seconds. Crowded carriages did not reduce helping at all.
Both findings are real. They do not contradict each other so much as answer different questions, and the difference lies almost entirely in how each study was built. That is the point most popular accounts of psychology miss.
The design of a study decides what its result can mean. A headline reports what was found; the method reports what the finding is entitled to claim.
Reading a psychological claim by its method means asking three plain questions. What was compared with what? Who took part? And what happened when someone else tried again?
Those questions are not a technical appendix to psychology. They are the literacy itself.
The comparison is the claim
The intercom study, by John Darley and Bibb Latané, is persuasive for a narrow and precise reason. The "victim" was a recording, identical for every participant. The only thing that changed was how many other listeners the participant believed were present: none, one or four.
When one factor moves and everything else holds still, a difference in behaviour can be traced to that factor. This is controlled comparison, and it is the only thing that licenses the word "cause".
Martin Seligman and Steven Maier used the same logic in their learned helplessness experiments. One dog could stop shocks by pressing a panel; a second, yoked to the first, received exactly the same shocks with no control; a third received none. Later, most dogs from the no-control group lay down and endured shocks they could have escaped.
The yoking is the argument. Because two animals received identical shocks, the difference between them can only be control itself.
The implication cuts both ways. Control buys certainty about cause inside the conditions that were fixed, and nothing more.
The intercom booths removed the very things the subway supplied: an unmistakable emergency and bystanders who could see each other. The subway gave up control and gained a real carriage. A lab experiment shows cause; a field study shows reach, and neither can do the other's job.
What people cannot tell you
A well-chosen comparison does more than establish cause. It can reveal processes that the people being studied cannot see in themselves, which matters because the most obvious way to find out why someone acted is to ask them.
Richard Nisbett and Timothy Wilson reviewed evidence that this rarely works. In one study, shoppers compared four identical pairs of nylon stockings laid out in a row and picked the best. The right-most pair was chosen almost four times as often as the left-most.
Nobody mentioned position. People cited knit and sheerness, and denied that position could matter. In a related experiment reported in the same paper, people's accounts of what had swayed them matched the guesses of observers who had never taken part.
The design works because the stockings were identical, so any preference had to come from something other than quality. The shoppers' reasons were plausible theories about themselves, not reports of what had moved them. Self-report is a theory, not an observation.
This is why so many durable findings in psychology rest on tasks rather than testimony. John Ridley Stroop timed people naming the ink colour of words; when "RED" was printed in green, naming slowed, because reading runs automatically and competes with the slower task. George Sperling found that people shown a flash of 12 letters could report about four and a half, yet a tone cueing a single row showed that most of the grid had briefly been available. In each case the method went around the participant's own account and measured the process directly.
The sample travels with the result
Comparison settles what caused what. It says nothing about for whom. Who was studied limits who the result applies to, and the selection of a sample is carried silently into every conclusion drawn from it.
The marshmallow test shows how.
Walter Mischel's studies at Stanford's Bing Nursery School offered preschoolers one treat now or two if they waited, often for 15 minutes. Follow-ups linked longer waiting to later academic outcomes, and the test became a symbol of willpower. In 2018, Watts, Duncan and Quan tested the idea with over 900 children from a more diverse sample. The association with achievement at 15 was about half the original size, and shrank much further once family background and early cognitive ability were taken into account.
The lesson is not that self-control is unimportant. It is that a nursery on a university campus is a particular population, and that waiting for a treat was tangled up with the advantages those children already had. Lewis Terman's gifted children, around 1,500 Californians chosen partly by teacher nomination, carry the same warning: none became a Nobel laureate, while two boys who were tested and not selected, William Shockley and Luis Alvarez, both did.
The contrast case is instructive. The Dunedin study enrolled 1,037 people born in 1972–73 in Dunedin, New Zealand, nearly every child in the city, and very few dropped out. Its finding that lower childhood self-control predicted worse health, finances and criminal records by 32 held after accounting for IQ and social class. The claim is similar; the sampling makes it far harder to dismiss.
One study is a rumour
The final question, whether a result survived re-testing, is where method stops being an academic nicety and becomes the difference between knowledge and folklore.
In 1998, Roy Baumeister and colleagues had hungry students resist chocolate cookies and eat radishes instead. On unsolvable puzzles afterwards, radish eaters gave up after about 8 minutes, against about 19 for those who had eaten cookies. The idea that self-control draws on a limited fuel spread through hundreds of studies. In 2016, a registered replication across 23 labs and more than 2,000 participants, following an agreed protocol, found an effect close to zero.
The larger picture came a year earlier. The Open Science Collaboration, 270 researchers, repeated 100 studies from three leading journals.
Of the originals, 97% had reported statistically significant results. Only about 36% of the replications did, and effect sizes averaged about half. A single study is weak evidence, however famous.
Yet the same scrutiny that sank ego depletion confirmed other classics. Hermann Ebbinghaus, memorising nonsense syllables as his own only participant in 1885, found that forgetting is steep at first and then levels off. A 2015 replication reproduced his curve closely.
The framing effect, Pavlov's conditioning and the Stroop effect have held up just as firmly. Replication sorts the durable from the lucky, and it does so without regard to reputation.
Reading the method first
Put the three questions together and a different picture of psychology emerges. Fame and reliability turn out to be almost unrelated. The Stanford Prison Experiment became one of the best-known studies in the field without ever being published in a peer-reviewed journal as a full experiment, and archival work later showed that guards had been coached to be tough. Its fame came from a story; its weakness was visible in its design all along.
That is why psychology's reforms belong alongside its findings rather than behind them. Preregistration spread after Daryl Bem's 2011 experiments on precognition passed standard analyses, which showed that the standards allowed too much flexibility; registered reports and multi-lab replications followed the 100-study project. The discipline learnt about its own methods the same way it learns about anything: by testing them.
The durable habit is to read a claim backwards, from its method to its headline. Before accepting that crowds fail to help, that willpower runs out or that waiting for a marshmallow predicts a life, ask the question that separates a result from a rumour: how would they know that? The answer is always in the design, and it usually says less, and more precisely, than the headline did.