Two Transcripts

Here are two transcripts from talks by Lancet authors in 2007: Gilbert Burnham speaking at MIT (link and pdf) and Les Roberts in a radio interview (link and pdf). Comments:

1) These transcripts were done by my interns. They did their best, but if you decide to quote from these, you should check the quote against the original source as well.

2) I have already highlighted (here and here) some of the more important parts of Burnham's talk. The main benefit of this version is that includes copies of his slides, making it much easier to follow the talk. On page 16, we have a slide showing that the confidence intervals for excess deaths including Falluja would be much wider (and include 0), just as I have been arguing for two years now. Pages 21-24 highlight a different version of the sampling plan than is described in the paper. Burnham claims that they did not restrict the sample to streets that crossed their main streets. Instead, they made a list of "all the residential streets that either crossed it or were in that immediate area." This is just gibberish.

First, if this was what they actually did, why didn't they describe it that way in the article? Second, given the time constraints, there was no way that the teams had enough time to list all such side streets. Third, even if the interviewers did do it this way, the problem of Main Street Bias would still exist, except it would be more Center Of Town Bias. Some side streets are in the "immediate area" of just one main street (or often in the area of none) and other side streets (especially those toward the center of a town or neighborhood) are near more than one. The later are much more likely to be included in the sample.

3) I do not think that the Roberts interview is nearly as interesting, although it is amusing to note how he makes fun of "economists" (read: Spagat) and "political scientists" (read: me) early in the talk. I think that his claims (page 7ff) that the Lancet surveys are consistent with UNDP results from 2004 are not correct either, but I have not been following that dispute closely enough to comment more.


4) I also liked this statement from Roberts about the fact that 2/3 of the violent deaths from L1 were in Falluja.

So we, in 2004, came out with an estimate that we thought a 100,000 excess deaths
had occurred. When we did that, we set the Fallujah numbers aside, we didn't think they were wrong, but the vast, as you say 2/3 of all the violent deaths we found were there in this one 30 house cluster in Fallujah, and if in 2004, the fall of 2004, you had been watching the news this would have been very plausible. First, there was that big, incursion into Fallujah into I think April, in which because the press coverage was so negative for the Americans they ended up backing off and pulling back out, and in the month or two months before we went in the city had been surrounded by American forces and was being shelled on a daily basis.

It is absurd to think that US forces were lobby artillery shells into Falluja on a daily basis in July, August or September of 2004.

UPDATE: A reader adds these comments on the Roberts interview.

For me the real shocker is that he claims multiple times that the BBC poll found 17% of households "lost a household member to violence".

Also, he says that that there is "overwhelming" statistical evidence that 30 clusters is enough. Where is the evidence?

On sampling he actually describes the procedure as originally described in the Lancet. You might want to point out the contradiction with Burnham's version that you already treat.

He misrepresents the L1 sampling procedures as standard when they were anything but. It's worth another look at the L1 paper on this. They paired governorates that they claimed with no evidence were similarly violent then they randomly selected one of each pair. Definitely not standard. They then improperly ignored this when calculating their CI's.

He admits that Lafta "didn't really understand all the fancy statistics and sampling methods…"

Just complete gobledegook on how visiting more clusters could only raise his estimates.

Totally misrepresents ILCS. This boils down to the same thing as was in your previous post on the IFHS as an overestimate. Pedersen wrote a paper saying that all surveys underestimate (I think this is probably wrong but OK) so he raises his estimate. But he points out explicitly that the argument about surveys underestimating applies just as much to L1 as it does to the ILCS. So Roberts tries to compare and adjusted ILCS to an unadjusted L1. His 37,000 figure isn't right either but it's not egregiously low.

The he says "most of the bodies coming into morgues are not from natural causes but from violence". That's like saying most deaths investigated by the police are violent. Does that prove that most deaths are violent? The morgues get only suspicious deaths and unclaimed bodies.

UPDATE 2009-02-24: Recent events have made a particular section of Burnham's talk highly relevant. The transcript around page 35 of the pdf is not exactly correct. Here is a better transcription.

There were limitations to record keeping; we had some criticism that we could not produce a record showing which households were visited and who names of people were, and so forth.

We intentionally did not record that, because we felt that if the team were stopped at a checkpoint, of which there are lots of checkpoints, and the records were gone through, some of you may have had this experience, where you stop at a checkpoint, people go through all your papers, read everything, and they find certain neighborhoods. That might have increased risk, which we didn't want to do.

There can be no doubt that Burnham told the MIT audience that the survey team did not record names. But we now know that they did.