AP Story on JSM Meeting

Here is the AP story by Paul Foy on the JSM session about the Lancet surveys.

The courtroom-style questioning came in a packed ballroom at the world's largest gathering of statisticians.

On the hot seat: a globe-trotting researcher who says his team's surveys of Iraqi households projected nearly 655,000 had died in the war as of July 2006, a number still ten times higher than conventional estimates.

Leslie F. Roberts and others from John Hopkins University took accounts of births and deaths in some Iraqi households to estimate that the country's death rate had more than doubled after the 2003 invasion.

Number crunchers this week quibbled with Roberts's survey methods and blasted his refusal to release all his raw data for scrutiny -- or any data to his worst critics. Some discounted him as an advocate for world peace, although none could find a major flaw in his surveys or analysis.

"Most of the criticism I heard was carping," said Stephen Fienberg, a professor of statistics at Carnegie Mellon University in Pittsburgh. "I thought the surveys were pretty good for non-statisticians."

Roberts, an epidemiologist, said he's opening a new front in the study of public health hazards: war. He has conducted about 30 mortality studies since 1990 in conflicts around the globe, including the Congo, where he was similarly accused of exaggerating war-related deaths.

Roberts organized two surveys of mortality in Iraqi households that were published last October in Britain's premier medical journal, The Lancet. He acknowledged that the timing was meant to influence midterm U.S. elections.

"It puts you in a position where you are going to get attacked," said Fritz Scheuren, a senior fellow at the University of Chicago's National Opinion Research Center, who is trying to organize another Iraqi survey to see if he can match Roberts's results.

Scheuren, the American Statistical Association's former president, said he couldn't find anything wrong with The Lancet surveys.

He complained, however, that he wasn't able to get Roberts to reveal which of his Iraqi surveyors conducted which surveys, information that could reveal any bias in workers who compile consistently implausible results.

Roberts, now a lecturer at Columbia University, said he won't release the researchers' identities for fear of exposing them to death or retaliation. The new Iraqi government has strongly disputed the findings.

The estimate was not meant to be precise, but practically every health statistic in America comes from similar sampling techniques, Roberts and his team said.

Scheuren said it's OK for a scientist to be an advocate for peace but Roberts's work deserves more scrutiny because of it.

"It happens that war is political," Roberts responded Wednesday after his grilling at a convention of more than 5,000 statisticians. "So I think those criticisms are valid. I am swimming in a much more political world that these statisticians."

In The Lancet article, Roberts and other researchers produced eight pages of text, maps, charts and many figures, but the number was the show-stopper: 654,965 war-related deaths, from the 2003 invasion of Iraq to last summer.

The estimate covered everything from battlefield casualties to civilians dying for lack of routine medical care. President Bush last fall said the estimate wasn't credible.

"Do I believe 650,000? No," Fienberg said. "Do I believe a lot of people are being killed? Yes."

Roberts and the statisticians agree that estimates of as few as 66,000 war-related Iraqi civilian deaths are based on sporadic media reports or incomplete accounts from hospitals and morgues.

Other Iraqis end up in mass graves or are promptly and randomly buried by relatives under Muslim law, Roberts said.

He said he personally supervised a 2004 survey of mortality in Iraqi households, slipping into the country from Jordan on the floor of a sport utility vehicle with $20,000 hidden in his shoes. It was too dangerous for him to return in 2006, but he said he had a trained team of Iraqi doctors in white coats ready for that door-to-door survey.

In all, the researchers questioned 2,850 randomly selected households, and Roberts said both surveys confirmed a soaring death rate after the invasion. Their short questionnaire sought to find out how many family members were born or had died since Jan. 1, 2002, establishing a baseline for a post-invasion death rate.

"Verifying these numbers wouldn't be too difficult, so why isn't it being done?" asked Jana Asher, a graduate student at Carnegie Mellon University, who dissected Roberts's surveys and found little fault with them.

Has Roberts denied the claim that the timing of the report was "meant to influence midterm U.S. elections."? Not that I have seen. See some discussion on Deltoid.