Next Steps
Assume for a second that you think some of the data in L2 is either fraudulent or a product of a corrupted survey process. What data would you want to look at from IFHS to test that hypothesis?
The problem is that L2 supporters can reasonably quibble with how IFHS "adjusts" for clusters that were too dangerous to visit. IFHS reports that:
(It is not clear to me why the authors did not perform a similar adjustment because of the missing clusters in Nineveh and Wasit. Perhaps the number of missing clusters was too small to matter? Perhaps the IBC data was not detailed enough to work with?)
Anyway, reasonable people will disagree over whether or not the IFHS adjustments for this problem were too small (L2 supporters) or too large (IBC supporters). On my list of to-dos is trying to calculate what the violent death estimate is without the adjustment. The trick to avoiding the whole morass is just to ignore these governorates altogether. Instead, throw out these 4, plus the 3 in Kurdistan (where everyone agrees things have been peaceful) and focus on the 11 others. Or just the subset of these 11 in which L2 (implausibly) reports extremely high violent mortality.
The beauty of this approach is that the IFHS estimates for these "complete" governorates require no adjustments. No clusters were skipped. Every household was checked. There is no good reason for the IFHS and L2 estimates to be that different. The confidence intervals for the IFHS will be much narrower because there is no extra uncertainty associated with the adjustment for missing clusters. Also, because we are ignoring Kurdistan, we will get a much more focussed look at the differences between IFHS and L2.
And, back of the envelope, those differences will be huge. As the IFHS authors note:
In other words, the big differences between L2 and IFHS are not in Baghdad. (In fact, and also on the to-do list, it is not clear to me that L2 and IFHS disagree that much about Baghdad, especially if we throw-out the deeply suspect results from cluster 33 in L2.) So, the disagreement in the remaining clusters will be large. Combine a large raw difference with narrow confidence intervals for the IFHS estimates, and you have a recipe for, as the Marxists say, heightening the contradictions.
The problem is that L2 supporters can reasonably quibble with how IFHS "adjusts" for clusters that were too dangerous to visit. IFHS reports that:
Of the 1086 originally selected clusters, 115 (10.6%) were not visited because of problems with security. These clusters were located in Anbar (61.7% of the unvisited clusters), Baghdad (26.9%), Nineveh (10.4%), and Wasit (0.8%). Since past mortality is likely to be higher in these clusters than in those that were visited during the IFHS, we imputed mortality figures for the missing clusters in Anbar and Baghdad with the use of information from the Iraq Body Count on the distribution of deaths among provinces to estimate the ratio of rates of death in these areas to those in other provinces with high death rates.
(It is not clear to me why the authors did not perform a similar adjustment because of the missing clusters in Nineveh and Wasit. Perhaps the number of missing clusters was too small to matter? Perhaps the IBC data was not detailed enough to work with?)
Anyway, reasonable people will disagree over whether or not the IFHS adjustments for this problem were too small (L2 supporters) or too large (IBC supporters). On my list of to-dos is trying to calculate what the violent death estimate is without the adjustment. The trick to avoiding the whole morass is just to ignore these governorates altogether. Instead, throw out these 4, plus the 3 in Kurdistan (where everyone agrees things have been peaceful) and focus on the 11 others. Or just the subset of these 11 in which L2 (implausibly) reports extremely high violent mortality.
The beauty of this approach is that the IFHS estimates for these "complete" governorates require no adjustments. No clusters were skipped. Every household was checked. There is no good reason for the IFHS and L2 estimates to be that different. The confidence intervals for the IFHS will be much narrower because there is no extra uncertainty associated with the adjustment for missing clusters. Also, because we are ignoring Kurdistan, we will get a much more focussed look at the differences between IFHS and L2.
And, back of the envelope, those differences will be huge. As the IFHS authors note:
All three sources agreed on the low mortality in Kurdistan. Of all the violent deaths occurring in Iraq, the proportion in Baghdad was 54% in the IFHS, 60% in the Iraq Body Count, and only 26% in the study by Burnham et al.
In other words, the big differences between L2 and IFHS are not in Baghdad. (In fact, and also on the to-do list, it is not clear to me that L2 and IFHS disagree that much about Baghdad, especially if we throw-out the deeply suspect results from cluster 33 in L2.) So, the disagreement in the remaining clusters will be large. Combine a large raw difference with narrow confidence intervals for the IFHS estimates, and you have a recipe for, as the Marxists say, heightening the contradictions.
posted by David Kane at 4:25 PM
0 Comments:
Post a Comment
<< Home