Truther

Les Roberts thinks that critics who question his studies are the same as those who deny that 3,000 people died on 9/11. Really.

And another thought is that -- quite unrelated -- if someone said in the 9/11 attacks, “I think only 200 or 300 people really died,” we would be really, really upset. And I think in the long view, the danger of discarding this study, if it’s correct, is that, at a moment when we as a society should be showing contrition, our leaders have essentially expressed indifference to an extraordinary level of suffering. And that’s just the wrong message in terms of either our long-term security or peace in the Middle East.

Is this better or worse than his other analogy of Lancet critics as Holocaust deniers?

Details on Falluja

The L1 authors consistently maintain that Falluja was not an outlier in the sense that it should be discarded. Instead, it was representative of many parts of Iraq. This is from a letter they wrote to the Independent.

Our study found that violence was widespread and up 58-fold after the invasion; that from 32 of the neighbourhoods we visited we estimated 98,000 excess deaths; and that from the sample of the most war-torn communities represented by 30 households in Fallujah more people had probably died than in all of the rest of the country combined.

Fallujah is the only insight into those cities experiencing extreme violence (ie Ramadi, Tallafar, Fallujah, Najaf); all the others were passed over in our sample by random chance. If the Fallujah duster is representative, there were about 200,000 excess deaths above the 98,000.

Perhaps Fallujah is so unique that it represents only Fallujah, implying that it represents only 50-70,000 additional deaths. There is a tiny chance that the neighborhood we visited in Fallujah was worse than the average experience, and only corresponds with a couple of tens of thousands of deaths. We also explain why, given study limitations, our estimate is likely to be low. Therefore, when taken in total, we concluded that the civilian death toll was at least around 100,000 and probably higher, not between 8,000 and l94,000 as Mr. Straw states.

I am still a little hazy on where the 50-70,000 number comes from and what it implies for the population of Falluja.

War Crime

The antics of Lancet editor Richard Horton are endless fun. Here he describes the invasion of Iraq as a "monstrous war crime."

This Labour government, which includes Gordon Brown as much as it does Tony Blair, is party to a war crime of monstrous proportions. Yet our political consensus prevents any judicial or civil society response. Britain is paralysed by its own indifference.

At a time when we are celebrating our enlightened abolition of slavery 200 years ago, we are continuing to commit one of the worst international abuses of human rights of the past half-century. It is inexplicable how we allowed this to happen. It is inexplicable why we are not demanding this government's mass resignation.

Two hundred years from now, the Iraq war will be mourned as the moment when Britain violated its delicate democratic constitution and joined the ranks of nations that use extreme pre-emptive killing as a tactic of foreign policy. Some anniversary that will be.

This is consistent with what Roberts occasionally says (can't find a link just now) about whether or not he would ever release detailed data from L1. He would, if the International Criminal Court needed the evidence for war crime prosecutions, presumably against civilians leaders like Bush and Blair.

Amer and Scheuren

One of the more annoying tendencies of Lancet defenders is that they refuse to recognize the breadth of academic criticism against the Lancet papers. Over time, I hope to gather some of that criticism together. Here is one example.

Several ASA statisticians have gone to Iraq since 2003—Jana Asher and Mary Gray, to mention two. An article by Asher about her experience appeared in the December 2006 issue of Amstat News; Gray’s work in Iraqi Kurdistan can be found in the April 2007 issue. So, the involvement of statisticians in Iraq is not unusual.

The recent controversy over the estimated Iraq war deaths that appeared in the October 11, 2006, issue of The Lancet has drawn considerable statistical attention to the situation in Iraq, as the results in the article did not appear credible. As mentioned by Gray, a follow-up to The Lancet piece occurred at a February session of the Washington Statistical Society (WSS), organized by Wendy Rotz, with talks by Asher and David Marker. The jury is still out on The Lancet results. Statisticians must assess the process used, not the outcome. But there is a lot wrong with the process, leading to the conjecture that the results are most probably wrong.

Indeed. Do these authors have standing for believing that the Lancet results are "probably wrong?" Judge for yourself.

Safaa Amer holds her PhD in statistics from Oregon State University. ... Fritz Scheuren is a vice president for statistics, who works with Amer at NORC at the University of Chicago. He put together the What Is a Survey? booklet some years ago. In his 2005 ASA presidential speech (March 2007, JASA), he advocated a pro bono role for statisticians.

I look forward to reading Daniel Davies thoughts on why Amer and Scheuren are just another two members of the ever-expanding list of anti-Lance cargo culters. Time to get those boots on!

Missing Data

I have been having fun on Deltoid recently (here and here). One annoying issue is whether or not the method by which SG replicated the L1 CMR estimates is obviously correct and/or the only reasonable approach. I don't think it is. (Given that it matches the formula in the spreadsheet distributed by Les Roberts, I am happy to assume that it matches the approach used in the paper.) Although I am a layman when it comes to demography, it seems obvious that any statistican would question whether just adding up all the deaths and dividing by person-months is the best way to estimate the crude mortality rate for Iraq. Such a proceedure ignores the fact that non-response varies across clusters.

Consider a simple example in which you have two clusters with 50 attempted interviews in each using a one year look-back period. In cluster A, you interview all 50 households. There are 10 people in each house and a total of 20 deaths. The CMR is cluster A is then 4% (20 deaths divided by 500 person-years). But, in cluster B, only 10 households agree to be interviewed. The other 40 refuse. There are also 10 people in each of the 10 households. There is one death, giving a CMR of 1% for cluster B.

Question: What is the best estimate of the CMR for the whole country given this data from two clusters? Answer: It depends! Certainly, the formula cited as obvious by SG, Robert Chung and others at Deltoid is not clearly the best answer (although I agree that it is a reasonable one). To see why, note that this formula just adds up the total deaths (20 + 1 = 21) and divides by the total person years in the population that agreed to be interviewed (500 + 100 = 600) to give a CMR of 3.5% (21/600). In other words, the overall estimate is very close to the estimate for cluster A because the actual sample size from cluster A is so much larger than that for cluster B even though the sampling plan called for equal sample sizes.

Ignoring non-response causes you to weigh clusters with higher-response rates more heavily even though, a priori, there is no particularly good reason to do so.

A different approach would be to treat the two cluster as independent and just average the resulting CMRs from each cluster. Such an approach would give 2.5%. Now, the difference between 2.5% and 3.5% isn't that big. Indeed, the differences in the two approaches for the Lancet data are even smaller. But the idea that one is clearly right and the other wrong is just stupid. It all depends on what you think the cause of the non-response is.

(A further complication is that household sizes may differ across clusters. In that case, it makes sense to either sum populations or to weight the clusters by the total population in each cluster. But the main point I am making here has to do with non-response.)