ILCS Response Rates

A common response to concerns about the high response rate achieved in Lancet II is to note that the Iraq Living Conditions Survey 2004 (ILCS) reported a response rate of 98.5%. The technical sophistication and competence of ILCS has been universally praised.

In each governorate, 1,100 households were selected for interview, with the exception of Baghdad, where 3,300 households were selected. The sample thus consisted of 22,000 households. Of these, 21,668 were actually interviewed.

In fact, the response rate for ILCS is higher than that for Lancet II (98.3%). So, what is the problem?

The issue is that ILCS was conducted in a much more thorough fashion than Lancet II.

COSIT staff were extensively trained in implementing the survey tool by researchers from the Fafo AIS. The first round of training took place in Amman, Jordan during the first three weeks of February 2003. Core staff from COSIT’s offices in each governorate were present, in addition to administrative staff from the headquarters in Baghdad. Training of local staff was subsequently conducted at six locations within Iraq during the first two weeks of March 2003 by COSIT’s core staff under supervision from Fafo.

Fieldwork started on March 22, 2004, and was completed by May 25, 2004. Data collection in the Governorates of Erbil and Dahouk were implemented and completed in August 2004.

After each selected PSU had been mapped and listed, interviewers were sent to the 10 selected households. Interviewers were organized in teams of five, with individual supervisors who continuously provided guidance and checked the quality of all incoming interviews. When necessary, interviewers were sent back to the households to reconfirm information. Furthermore, supervisors from COSIT’s headquarters in Baghdad and Fafo staff also visited the interviewer teams.

Upon completion of the interviews, the information was sent to the governorate office for registration and inspection, then to the Baghdad main office for coding and data entry. During the data entry process, extensive quality control was implemented, and questionnaires were sent back to the field for re-interviewing or update both by COSIT’s Baghdad office and by Fafo headquarters in Oslo.

Completed data files were continuously sent to Fafo’s headquarters in Oslo, Norway, where further quality checks were implemented. In instances where problems arose, direct communication was made with COSIT. Several times during the fieldwork, COSIT arranged meetings with its offices’ heads in order to inform them of problems that had surfaced and resolve them.

In other words, ILCS interviewers went back to the sample households again and again and again. This is quite different from the procedure in Lancet II. In that case, a cluster was visited on just one day. In fact, it appears that houses were just checked one time. What good fortune that there was almost always someone (head of house or spouse) at home!

What we need, obviously, is more information about the initial ICLS samples. How many households were present for this first interview? How many immediately agreed to participate? Only this level of detail will tell us if the final ILCS response rate is relevant in evaluating the reliability of the Lancet II sample.