Misleading Statements on Unique Identifiers

I had assumed that it would be easy to find places where Roberts/Burnham claimed that names were not collected. Turns out that it isn't. Looks like they knew that this fact might eventually come out so they tried to mislead without lying. Consider their letter to Science.

Those who work in conflict situations know that checkpoints often scrutinize written materials carried by those stopped, and their purpose may be questioned. Unique identifiers, such as neighborhoods, streets, and houses, would pose a risk not only to those in survey locations, but also to the survey teams. Protection of human subjects is always paramount in field research. Not including unique identifiers was specified in the approval the study received from the Johns Hopkins Bloomberg School of Public Health Committee on Human Research. At no time did the teams “destroy” details, as Bohannon contends. Not recording unique identifiers does not compromise the validity of our results.

Were Burnham/Roberts lying? Tough to say! I (and everyone else who read those words two years ago) assumed that the names of the participants were not collected. (We now know that at least some full names were recorded.) But the above passage is written in such a way as to give the impression that no names were collected --- "Not including unique identifiers was specified in the approval the study received" --- without explicitly stating that this had been done. Clever!

Recall how the paper itself handled the issue.

The survey purpose was explained to the head of household or spouse, and oral consent was obtained. Participants were assured that no unique identifiers would be gathered.

Again, this could be a truthful statement. Who knows what the interviewers said to the participants? But it completely misleads the reader into thinking that names had not been collected when, in fact, they had been collected.

So, hard core Lancet defenders will insist that no correction is necessary. Then why is Johns Hopkins insisting that one be made?

The paper in The Lancet incorrectly stated that identifying data were not collected. An erratum will be submitted to The Lancet to correct the text of the 2006 paper on this point.

If you agree with Hopkins that a correction is necessary, then you should also believe that Roberts/Burnham owe an apology/correction to, among others, the readers of Science. They were just as likely to have been misled.

But also consider this June 2007 letter to Science from Burnham, Roberts and Doocy.

At the time the study was published, Iraqi colleagues requested that we delay release, as they were very fearful that somehow streets, houses, and neighborhoods might be identified through the data with severe consequences. We agreed to wait for 6 months and have now made the data available.

From the beginning, we have taken the position that protecting participants is paramount, and thus we will not be releasing any identifiers below the level of governorate. The demand by some for street names seems to arise from the erroneous belief that only main streets were sampled, when in fact, where there were residential streets that did not intersect with the selected commercial street, these too were included in the sampling frame for identification of the start house. In any event, our interviewers reported that most, although not all, violent deaths occurred away from the location of residence.

Does this make much sense? To the extent that their "Iraqi colleagues" were "very fearful," it would not be because of specific neighborhoods being identified via statistical magic but because the full names of participants could be released. That's the real problem. By pretending otherwise, Burnham/Roberts/Doocy continued to mislead all of us into thinking that the worst conceivable data release (from the point of view of participant safety) might allow identification of a specific street when, in fact, the worst possible release would involve the actual names of participants. Again, there is nothing in the above that is literally a lie --- Who knows what their "Iraqi colleagues" told them? --- but there is no doubt that readers were misled, again, into thinking that names had not been recorded.