UK Times Article

Interesting article from the UK Times Online.

The statistics made headlines all over the world when they were published in The Lancet in October last year. More than 650,000 Iraqis – one in 40 of the population – had died as a result of the American-led invasion in 2003. The vast majority of these “excess” deaths (deaths over and above what would have been expected in the absence of the occupation) were violent. The victims, both civilians and combatants, had fallen prey to airstrikes, car bombs and gunfire.

Body counts in conflict zones are assumed to be ballpark – hospitals, record offices and mortuaries rarely operate smoothly in war – but this was ten times any other estimate. Iraq Body Count, an antiwar web-based charity that monitors news sources, put the civilian death toll for the same period at just under 50,000, broadly similar to that estimated by the United Nations Development Agency.

The implication of the Lancet study, which involved Iraqi doctors knocking on doors and asking residents about recent deaths in the household, was that Iraqis were being killed on an horrific scale. The controversy has deepened rather than evaporated. Several academics have tried to find out how the Lancet study was conducted; none regards their queries as having been addressed satisfactorily. Researchers contacted by The Times talk of unreturned e-mails or phone calls, or of being sent information that raises fresh doubts.

Iraq Body Count says there is “considerable cause for scepticism” and has complained that its figures had been misleadingly cited in the The Lancet as supporting evidence.

One critic is Professor Michael Spagat, a statistician from Royal Holloway College, University of London. He and colleagues at Oxford University point to the possibility of “main street bias” – that people living near major thoroughfares are more at risk from car bombs and other urban menaces. Thus, the figures arrived at were likely to exceed the true number. The Lancet study authors initially told The Times that “there was no main street bias” and later amended their reply to “no evidence of a main street bias”.

Professor Spagat says the Lancet paper contains misrepresentations of mortality figures suggested by other organisations, an inaccurate graph, the use of the word “casualties” to mean deaths rather than deaths plus injuries, and the perplexing finding that child deaths have fallen. Using the “three-to-one rule” – the idea that for every death, there are three injuries – there should be close to two million Iraqis seeking hospital treatment, which does not tally with hospital reports.

“The authors ignore contrary evidence, cherry-pick and manipulate supporting evidence and evade inconvenient questions,” contends Professor Spagat, who believes the paper was poorly reviewed. “They published a sampling methodology that can overestimate deaths by a wide margin but respond to criticism by claiming that they did not actually follow the procedures that they stated.” The paper had “no scientific standing”. Did he rule out the possibility of fraud? “No.”

If you factor in politics, the heat increases. One of The Lancet authors, Dr Les Roberts, campaigned for a Democrat seat in the US House of Representatives and has spoken out against the war. Dr Richard Horton, editor of the The Lancet is also antiwar. He says: “I believe this paper was very thoroughly reviewed. Every piece of work we publish is criticised – and quite rightly too. No research is perfect. The best we can do is make sure we have as open, transparent and honest a debate as we can. Then we'll get as close to the truth as possible. That is why I was so disappointed many politicians rejected the findings of this paper before really thinking through the issues.”

Knocking on doors in a war zone can be a deadly thing to do. But active surveillance – going out and measuring something – is regarded as a necessary corrective to passive surveillance, which relies on reports of deaths (and, therefore, usually produces an underestimate).

Iraq Body Count relies on passive surveillance, counting civilian deaths from at least two independent reports from recognised newsgathering agencies and leading English-language newspapers ( The Times is included). So Professor Gilbert Burnham, Dr Les Roberts and Dr Shannon Doocy at the Centre for International Emergency, Disaster and Refugee Studies, Johns Hopkins Bloomberg School of Public Health, Maryland, decided to work through Iraqi doctors, who speak the language and know the territory.

They drafted in Professor Riyadh Lafta, at Al Mustansiriya University in Baghdad, as a co-author of the Lancet paper. Professor Lafta supervised eight doctors in 47 different towns across the country. In each town, says the paper, a main street was randomly selected, and a residential street crossing that main street was picked at random.

The doctors knocked on doors and asked residents how many people in that household had died. A person needed to have been living at that address for three months before a death for it to be included. It was deemed too risky to ask if the dead person was a combatant or civilian, but they did ask to see death certificates. More than nine out of ten interviewees, the Lancet paper claims, were able to produce death certificates. Out of 1,849 households contacted, only 15 refused to participate. From this survey, the epidemiologists estimated the number of Iraqis who died after the invasion as somewhere between 393,000 and 943,000. The headline figure became 650,000, of which 601,000 were violent deaths. Even the lowest figure would have raised eyebrows.

Dr Richard Garfield, an American academic who had collaborated with the authors on an earlier study, declined to join this one because he did not think that the risk to the interviewers was justifiable. Together with Professor Hans Rosling and Dr Johan Von Schreeb at the Karolinska Institute in Stockholm, Dr Garfield wrote to The Lancet to insist there must be a “substantial reporting error” because Burnham et al suggest that child deaths had dropped by two thirds since the invasion. The idea that war prevents children dying, Dr Garfield implies, points to something amiss.

Professor Burnham told The Times in an e-mail that he had “full confidence in Professor Lafta and full faith in his interviewers”, although he did not directly address the drop in child mortality. Dr Garfield also queries the high availability of death certificates. Why, he asks, did the team not simply approach whoever was issuing them to estimate mortality, instead of sending interviewers into a war zone?

Professor Rosling told The Times that interviewees may have reported family members as dead to conceal the fact that relatives were in hiding, had fled the country, or had joined the police or militia. Young men can also be associated with several households (as a son, a husband or brother), so the same death might have been reported several times.

Professor Rosling says that, despite e-mails, “the authors haven’t provided us with the information needed to validate what they did”. He would like to see a live blog set up for the authors and their critics so that the matter can be clarified.

Another critic is Dr Madelyn Hsaio-Rei Hicks, of the Institute of Psychiatry in London, who specialises in surveying communities in conflict. In her letter to The Lancet, she pointed out that it was unfeasible for the Iraqi interviewing team to have covered 40 households in a day, as claimed. She wrote: “Assuming continuous interviewing for ten hours despite 55C heat, this allows 15 minutes per interview, including walking between households, obtaining informed consent and death certificates.”

Does she think the interviews were done at all? Dr Hicks responds: “I’m sure some interviews have been done but until they can prove it I don’t see how they could have done the study in the way they describe.”

Professor Burnham says the doctors worked in pairs and that interviews “took about 20 minutes”. The journal Nature, however, alleged last week that one of the Iraqi interviewers contradicts this. Dr Hicks says: : “I have started to suspect that they [the American researchers] don’t actually know what the interviewing team did. The fact that they can’t rattle off basic information suggests they either don’t know or they don’t care.”

And the corpses? Professor Burnham says that, according to reports, mortuaries and cemeteries have run out of space. He says that the Iraqi team has asked for data to remain confidential because of “possible risks” to both interviewers and interviewees.