Timing Comments

Kevin Wagner sent in these comments on the timing issue.

There are at least three claims about the Lancet study's procedures and the feasibility of its methods to which Les Roberts has given answers that would entail he either was misinformed about some of the fundamental aspects of his survey or was willing to play fast and loose with his rhetoric to dissuade inquiry.

The first two instances come from an article by the journal Nature, a copy of which is cited here:

http://psychoanalystsopposewar.org/blog/2007/03/01/nature-on-iraq-mortality-study/

1) "Roberts and Gilbert Burnham, also at Johns Hopkins, say local people were asked to identify pockets of homes away from the centre; the Iraqi interviewer says the team never worked with locals on this issue."

To my knowledge, this has never been addressed further by Roberts. It's an outright contradiction of the authors' claim with the interviewer's. It raises the questions of how much the study's authors accurately knew about their interviewers' behavior and also potential bias introduced from a failure to follow the study's methods on the parts of the interviewers.

2) "The US authors subsequently said that each team split into two pairs, a workload that is "doable", says Paul Spiegel, an epidemiologist at the United Nations High Commission for Refugees in Geneva, who carried out similar surveys in Kosovo and Ethiopia. After being asked by Nature whether even this system allowed enough time, author Les Roberts of Johns Hopkins said that the four individuals in a team often worked independently. But an Iraqi researcher involved in the data collection, who asked not to be named because he fears that press attention could make him the target of attacks, told Nature this never happened. Roberts later said that he had been referring to the procedure used in a 2004 mortality survey carried out in Iraq with the same team (L. Roberts et al. Lancet 364, 1857–1864; 2004)."

This clarification is problematic. Apparently, in response to a follow up question on the 2006 study, Roberts replies with something relevant only to 2004, i.e. that the interviewers worked independently. They did work independently in 2004 and did not [so he, Burnham and the interviewer claim] in 2006. Moreover, it is damaging to his case for the 2006 study's feasibility. In 2004, he had six effective interviewers. In 2006, he had four two-person teams, i.e., four effective interviewers. How would responding that he effectively had two more interviewers in 2004 than in 2006 prove that his 2006 timeframe was feasible?

3) Last is Roberts' response on to queries on the interview timeframe and his reply to Tim Lambert on same.

http://www.radioopensource.org/les-roberts-weighs-in-on-lancet-controversy/

He states:

"The two main criticisms which were in both the *Nature* article and *The Times* article are completely without merit. They said there wasn't enough time to have done the interviews. We had eight interviewers working ten hour days for 49 days, they had two hours in the field to ask each household five questions. They had time."

They had eight interviewers which he says worked 49 ten hour days. 8 interviewers x 10 hours x 49 days / 2000 houses in the sample = 1.98 hours.

However, as noted before, those 8 worked in two-person teams. 4x10x49/2000=.98.

In any case, how is the total time that was devoted to the entire project supposed to relate to the actual time they spent in the field? One to one correspondence? Every second they had for the project was spent in the field? Short of that, what does Roberts mean when he says they had 2 hours in the field per household?

http://scienceblogs.com/deltoid/2007/03/london_times_hatchet_job_on_la.php

Then Tim Lambert says he got a reply from Roberts in a post in the above blog entry:

"I asked Roberts and he says he agrees with Burnham about the interview length: 15-20 minutes."

So, again, what exactly is Roberts claiming when he says the interviewers had two hours in the field per household to ask five questions? How is this claim accurate or relevant? Interviews took 20 minutes, on the high side, according to Roberts. To what is "two hours in the field" supposed to relate? Did the interviewers really have an additional hour and 40 minutes per household in the field as his claim implies?

I am not sure how much I worry about this issue, especially since Burnham has told us that, on occasion, single interviewers would work alone (although we have no idea how often that occurred). It is further evidence that the US authors have trouble knowing exactly what the interviewers did. Related comments here.