Rolling Stone and the perils of confirmation bias

Last year, Rolling Stone published a blockbuster investigative report about rape and the University of Virginia. In "A Rape on Campus," reporter Sabrina Rubin Erdely painted a picture of administrative and cultural betrayal at UVA, with a central focal point: "Jackie," who alleged that she was gang raped at the fraternity Phi Kappa Psi. Jackie told Erdely that her friends discouraged her from reporting the attack, and that when she finally did go to the school's administration, they didn't support her at all. Erdely, who had been looking for a case to use as a framework to expose the "rape culture" on campuses, had found exactly what she wanted.

That was a problem. But as the Columbia School of Journalism reported after a four-month investigation of this fatally flawed Rolling Stone article, it was hardly the only problem. Nor was it the issue that reflected worst on Rolling Stone, which ended up in the strange position of endorsing both their policies and their personnel while promising not to produce another instance of imagineered fabulism in the future.

Erdely's search for her chosen narrative above all else, however, enabled the cascade of failures that followed. And it reveals the true perils of confirmation bias.

Subscribe to The Week

Escape your echo chamber. Get the facts behind the news, plus analysis from multiple perspectives.

SUBSCRIBE & SAVE

Sign up for The Week's Free Newsletters

From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.

From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.

Sign up

The story began last summer, Columbia's scathing post-mortem notes, when Erdely and Rolling Stone decided to do a feature story on the subject of rape on college campuses. This topic has provoked much public debate for a number of reasons, not least of which is the fact that activists have pushed a moral panic to create pressure for public action. This was enabled by the White House, which has pushed the widely debunked statistic that one in five women in America has been the victim of rape or attempted rape, as well as one in five women during their college years. In fact, the Department of Justice reported in December that the incidence of rape for women attending college between 1995 and 2013 was 6.1 per 1,000 people, and that it's actually a lower rate than in the general population, 7.6 per 1,000 people. On a percentage basis, the rate for rape among female college students is 0.61 percent, not 20 percent. Obviously, any rapes is too many. But the numbers matter.

The topic of rape is serious, and it is ready for serious investigative journalism. But it requires journalists to start at a point of skepticism of both sides. Instead, Erdely and Rolling Stone went into this project with their minds made up about what they would "report," and only needed a compelling story from a young woman to lend their predisposition some credence. "Erdely said she was searching for a single, emblematic college rape case," the Columbia review states, "that would show what it's like to be on campus now … where not only is rape so prevalent but also that there's this pervasive culture of sexual harassment/rape culture,' according to Erdely's notes of the conversation." In other words, Erdely and Rolling Stone were eager to feed and justify the moral panic rather than use journalism to see whether it was warranted at all.

In a press conference on Monday to review the report, Columbia Journalism School dean Steve Coll — a recipient of the Pulitzer Prize — warned about this practice. Responding to a question about "narrative journalism," Coll warned of its dangers, especially using the "emblematic anecdote" model. "How do you think about the shock narrative at the top of the story?" Coll wondered. "Are you coming there with an empirical state of mind, or are you coming there with an illustrative state of mind?"

The former might lead reporters in new and more truthful directions, but the latter leads to confirmation bias. "We talked in our report about confirmation bias," Columbia professor Sheila Coronel said in response to a question about the preconceived narrative, "about the tendency to see any facts in the case that fit a certain narrative or preconceived notions."

That criticism doesn't just apply to Erdely. Rolling Stone wanted this story, as Columbia's report makes clear, even when a number of red flags appeared before publication. Erdely's article got reviewed by a lot of journalists. They blew it. The narrative was so compelling, fact-checking chief Coco McPherson told Columbia's investigators, that the magazine's internal policies were ignored. "I one hundred percent do not think that the policies that we have in place failed," said McPherson. "I think decisions were made around those because of the subject matter."

In other words, the agenda trumped normal journalistic considerations. The magazine wanted an "emblematic narrative" to service the hype about campus rape, and didn't want to look too closely once Erdely found it. This explains the otherwise puzzling announcement by Wenner that no one would get fired from Rolling Stone for its most embarrassing journalistic collapse, not even Erdely, a decision that The Washington Post's Chris Cillizza called "terrible for journalism."

This decision raises another question. Just how many times has Rolling Stone decided that the agenda trumped the same journalistic policies they ignored for Erdely's story? The press asked Coll and Coronel whether they planned to review the magazine's other reporting for similar failures, to which they noted that Rolling Stone had not supplied them with the transcripts for any other story, and that it took four months to conduct this review.

However, reporters can certainly demand that Rolling Stone answer the question directly. If Wenner balks at cooperating, that would speak volumes about Rolling Stone's credibility for the future. And if reporters don't bother to ask, it will speak volumes about theirs in the present.