End Game

How will the controversy over the Lancet articles end? Good question. I can't predict the future, but here are some benchmarks for defining "victory," at least for we Lancet critics.

1) Both Roberts et al (2004) and Burnham et al (2006) are withdrawn by the Lancet. This is the primary goal that many of us are aiming for. If scientists don't police the accuracy of the scientific literature, then who will?

2) The Lancet authors are forced to be more transparent in their research. There is still some (small) chance that the Lancet papers are accurately, or at least not fraudulent. If the authors were to provide access to their data/methods to all researchers, then that would be a victory for replication and the scientific process. My primary motivation from the start has been to insistent on the importance of scientific work that is open to replication. That is the primary norm that I want to defend. If the Lancet authors don't have to share their data or explain their statistics, then why should any other scientist have to?

3) Censure of Roberts/Burnham by Johns Hopkins or (in the case of Roberts) Columbia. If leading research universities don't uphold the norms of open and transparent scientific inquiry, who will? The most interesting aspect of the recent censure of Burnham by AAPOR was news of an open investigation by Johns Hopkins.

In a highly unusual rebuke, the American Association for Public Opinion Research today said the author of a widely debated survey on "excess deaths" in Iraq had violated its code of professional ethics by refusing to disclose details of his work. The author's institution later disclosed to ABC News that it, too, is investigating the study.

AAPOR, in a statement, said that in an eight-month investigation, Gilbert Burnham, a faculty member at the Johns Hopkins Bloomberg School of Public Health, "repeatedly refused to make public essential facts about his research on civilian deaths in Iraq."

Hours later, the school itself disclosed its own investigation of the Iraq casualties report "to determine if any violation of the school's rules or guidelines for the conduct of research occurred." It said the review "is nearing completion."

...

The inquiry by the Johns Hopkins Bloomberg School was disclosed in an e-mail from Tim Parsons, the school's public affairs director, as follows:

"The level of civilian mortality in Iraq is a controversial subject. Questions have been raised regarding the findings and methodology of the 2006 Iraq mortality study conducted by Dr. Gilbert Burnham and published in The Lancet. The Johns Hopkins Bloomberg School of Public Health takes any allegation of scientific or professional misconduct very seriously. It believes that the correct forum for discussing the reported findings of the Lancet study and the general methodology that led to those findings is in the regular exchange of views in the scientific literature. The Bloomberg School of Public Health has undertaken a review of the study to determine if any violation of the school's rules or guidelines for the conduct of research occurred in the conduct of the study. That review is nearing completion and the school is unable to discuss the results at this time."

I hesitate to speculate on what Hopkins will conclude.

4) A change in policy by the Lancet to require authors to allow for replication of their work. More and more journals are moving in this direction and I expect the Lancet to follow suit at some point.

Will any of these come to pass? I don't know. But, heading into year 5 of this controversy, I am pleased with the progress that we have made so far.