In December 2014, Sean Woods received a call editors dread the most. A Rolling Stone reporter told him she no longer stood by her story of a horrible rape committed by frat boys at the University of Virginia. The feature, published a few weeks earlier, broke readership records at the iconic magazine. It caused a national uproar. It is not hard to see why. The story recounts the dramatic allegations of a young woman, lured by her date to a fraternity house, there to be gang-raped on broken glass by seven initiates. The line the story took was that the victim’s experience was emblematic of a ‘culture’ female students faced on campuses across the US. This heinous crime was compounded by the defensive, almost dismissive, response of University authorities.

Sabrina Erdely is an experienced contributor to Rolling Stone.  Her last story, ‘A Rape on Campus’, went through a fact checker, an editor and legal counsel.  Other writers, however, notably in the Washington Post, had no difficulty exposing her exposé. At best, it was shoddy, prejudicial journalism with the underlying claim still intact. Fair to middling, a teenage attention-seeker took Erdely in, when an application of elementary investigative reporting techniques would have detected the lies with ease. At worst, the student’s account was an inherently improbable hoax and the only reason Erdely recounted it at all was that it confirmed her ideological and ‘gothic’ expectation of how entitled, rich, male undergraduates were apt to behave.  As Cathy Young put it, Erdely’s story flowed from the pen of one possessed of “the troubling zealotry of advocates for whom believing rape claims is somewhat akin to a matter of religious faith”.

UVA

In due course, the story unraveled in newspapers and blogs across the US.  Rolling Stone approached the dean of the Columbia School of Journalism with a simple and acute brief.  Would Steve Coll help them understand how they had got things so terribly wrong? The review was published in early April 2015.  It found that the writer and editors were far too deferential to the single pseudonymous source, the supposed victim, Jackie.  Unwisely sensitive to her emotional well-being, Rolling Stone permitted Jackie to dictate the scope of the investigation and even their own questions to her.  Erdely further failed to verify crucial facts such as that the alleged rapist actually existed or that Jackie’s contemporaneous claims to friends matched what she was saying now.  Erdely’s breach of press protocol did not end there.  In what a Washington Post writer called an ‘unfathomable deceit’, Erdely and her editor misattributed dialogue and unfairly deprived the targets of her investigation of an opportunity to reply to intensely damaging accusations.

The Coll review has itself attracted edifying commentary about the pitfalls of narrative journalism and the shaming by ‘activists’ of those who pointed out the story’s glaring holes. Amidst the condemnation of Rolling Stone, there is a sense that this is a learning moment for the profession as a whole.

While some of the faults of the piece flow from Erdely’s unique and perhaps even malicious failings as the author, other errors are common to many in the profession.  The error that has attracted the least commentary so far is the idea that Jackie’s credibility was legitimately enhanced because her detailed story stayed consistent over its retellings to the journalist and because of her ‘confident’ demeanour during these narrations.  Many a reporter has fallen into these two holes before.

The Coll review states that Erdely “remembered being ‘a bit incredulous’ about the vividness of some of the details Jackie offered, such as the broken glass from the smashed table. Yet Jackie had been ‘confident, she was consistent.’”  Erdely credits the quality of consistency in Jackie’s versions of events in a later interview to Slate.  This is what convinced her.  All those details, staying the same, with every interview.

The fact-checker, who spent four hours on the phone reviewing Jackie’s story was also impressed by her vivid consistency. “She wasn’t just answering, ‘Yes, yes, yes,’ she was correcting me.”

In ascribing credibility to a story because it stays consistent, investigative journalists commit what amounts to a fallacy in evidentiary terms. Muckraking is the younger profession and journalists are well advised to take heed of the extreme caution, amounting to an exclusionary rule, with which the older profession of lawyering treats prior consistent statements. Contrary to common sense, when a witness repeats an accusation he has made on a prior occasion, this does not increase the credibility of the claim. That witness could be consistently lying or consistently telling the truth. Mere consistency does not tell us whether the scale is rigged or not.

Of course, if a witness is accused of recently fabricating a version, the fact that he has long been making the same claim is a fair rebuttal of this imputation.  An account given immediately or spontaneously after an event also carries a certain weight.  But judges and juries are usually spared the information that a complainant has, thereafter, repeated their allegations in detail.  This is because logically, self-corroboration, sheds no light on credibility.

Strange as it may seem at first glance, while prior consistency does not augment credibility, prior inconsistency does validly impeach or discredit a witness.  There certainly were material and conspicuous inconsistencies in Jackie’s story, if only Erdely had bothered to find them.

Demeanour is another notoriously unreliable measure of a person’s honesty. Studies consistently show that Erdely had odds little better than flipping a coin in discerning whether Jackie was telling the truth judging by her demeanour. Confidence means as little as being embarrassed when reporting a sexual assault. Indeed, Erdely reads confidence as increasing Jackie’s credibility during some of their interactions, while Jackie’s fear and reticence during other encounters does exactly the same thing. This underscores the unreliable nature of demeanour as an objective sign of truth-telling and reveals Erdely’s own determination to believe Jackie’s story, no matter, actually, how she told it.

None of this means that demeanour is worthless. Experienced journalists often have “a feeling” about a story.  The feeling arises from a source’s tears or smiles, their straightforward manner of speaking, co-operation, concessions and that elusive quality we call genuineness.  The most careful journalists though do not hand out the wreath of genuineness based on demeanour alone.  Even if reporters do not forensically analyse their structure of belief in these terms, they have also performed checks for contradiction, reliability of observation and bias.  They have thought really hard about the inherent plausibility of an allegation and sought corroboration for tales that go against the way the world ordinarily works. In the alchemy of investigative reporting many ingredients combine with a good demeanour to produce the ‘feeling’ that a source can be trusted.

There are huge risks, however, if the feeling of genuineness arises only from the inconsolable weeping or, alternatively, steady-eye and welcoming handshake of a source.  Extending such ill-founded credit to a source may well be a sign that the halo effect has kicked in or else its a case of an Erdelian confirmation bias about how bad the alleged wrong-doers must be.

Investigative reporting is an inherently contentious business.  If the target is important and well resourced, if the stakes are high, there will be significant, coordinated and biased blowback.  The greater the impact of a piece, the shriller the shrieks of indignation, the sterner the letters from lawyers and the greater sanctimony from scrutinizing commentators.

When that kind of heat arrives, it is far better to have forensically examined the facts of your story yourself before publication than to have your adversaries do so afterwards.  The Coll review of the Rolling Stone story of a campus rape that never happened has supplied useful new guidelines by which to avoid retraction and hefty lawsuits.  It turns out that old law books on evidence may be consulted too for additional, cross-over insights on how (not) to assess the credibility of a source.

This article first appeared in Opinion Nigeria