Air Strikes Before The War

If you believe the data from Burnham et al. (2006), you should conclude that approximately 2,000 Iraqis were killed by coalition air strikes before the war began in March 2003. How did I derive that number? And is it plausible?

In the main paper, Burnham et al. (2006) report on page 3 that there was one death due to "air strikes" in the pre-invasion period without clearly specifying whether or not this air strike was from US/coalition aircraft. On page 5, they write, "Violent deaths that were directly attributable to coalition forces or to air strikes were classified as coalition violent deaths." This seems reasonable since, after the start of the war, all aircraft are controlled by the coalition.

There were 82 total pre-invasion deaths recorded during the 14 months from January 2002 through March 2003. If the pre-invasion mortality rate was 5.5 per 1,000 and 1/82 were due to air strikes, then the air strike mortality rate was 6.7 per 100,000 per year. Using 26 million as an estimate for the total Iraqi population during the pre-invasion period, this would suggest a point estimate of approximately 2,000 ((5.5/1,000) * (1/82) * 26,000,000 * (14/12)). The Lancet reports a 95% confidence interval for the 5.5 per 1,000 mortality rate of 4.3 to 7.1. Plugging these numbers into the formula, would give us a range of 1,600 to 2,600. (This is, of course, not a good confidence interval for the number of air strike deaths since the fact that we only have one death means that a properly calculated interval would be much wider.) What can explain this result?

1) Perhaps there has been a coding/transcription error. It is certainly easy to imagine that someone could misread a date filled out by hand on a survey form.

2) Perhaps there was a mistake by a respondent. Three years after the fact, someone might think that his uncle had died in January 2003 instead of April 2003. Even trickier are deaths which happened on the first day or two of the war.

3) Perhaps this was a freak sample. Only a handful of non-soldiers were killed by coalition air strikes prior to March 2003, but, it just so happened that one of these unlucky families was included in the sample.

4) This could be an accurate representation of the truth. Even though there are almost no news stories (which I could find) documenting Iraqi deaths due to US air strikes prior to the war, this does not mean that thousands of deaths did not occur. We could have a very interesting finding on our hands. See Wikipedia for a description of Operation Southern Focus.

The operation was not publicly declared at the time, and was just said to be an intensification of the already-existing Operation Southern Watch. When it began, the United States Defense Department and CENTCOM stated that increasing numbers of bombings of Iraqi installations in the region were merely in response to more attacks by the air-defense forces of that country. The Iraqi no-fly zones had been patrolled continuously since the end of the 1991 Gulf War, and bombings by American and coalition fighter aircraft had taken place on a regular basis. However, Southern Focus saw many more engagements. Coalition forces responded to 651 attacks by dropping 606 bombs on 391 targets over the course of the operation.

So, perhaps there were approximately 2,000 Iraqis killed by coalition air strikes prior to the start of the war. But note that the survey requires victims to have lived in the surveyed home for 3 months prior to their deaths. Soldiers living away from home are specifically excluded.

If there were thousands of Iraqi civilians being killed by (mostly US) air strikes during 2002, it is hard to understand why Saddam Hussein would not broadcast that fact to the outside world.

5) This could be evidence of bias by the survey team. If they were trying to "find" as many deaths as possible caused by coalition forces, then they might be more likely to come up with highly unlikely data points like this one. If the questions that they asked were leading ones, if the people they interviewed were not really selected randomly, then such a response becomes more plausible.

Now, which of these explanations strikes you as most plausible, depends on your priors about the reliability of the survey teams. If you believe that they are highly trained professionals with no particular interest in the survey results, then you are most likely to attribute the estimate of 2,000 deaths to a transcription error or to suspect that the US pre-war bombing campaign was much more intense than most people realize. If you are suspicious about the motivations of the survey teams (or at least one of the individuals on those teams), then you would be more likely to believe that this data point is fraudulent.

But, for me, the key point is that Burnham et al refuse to release the underlying data, so it is impossible for any outsider to evaluate these explanations. It would be good to know if this was one of the deaths that the survey team asked to see a death certificate for and, if the team did ask, to know if a death certificate was available. For example, if this death occurred on March 17, 2003, then it is easy to understand how coalition air strikes might have been quite heavy on the day before the war "began." If this death happened in Baghdad (in which there are no reports of pre-invasion bombing) in July 2002, then it would seem to be evidence that something is wrong with the underlying data.

Being a suspicious person, I would wager that there was no death certificate available for this death or that the survey team did not even ask to see one. To prove me wrong, Burnham et al need only release the data. Why do they refuse to do so?