Gallup asks thousands of survey questions each year to answer some of the world's toughest and most sensitive questions on topics from health and education to wellbeing and politics. But before it does that, Gallup -- like many other researchers -- uses cognitive interviewing, as well as several other tools, to ensure the questions it is going to ask are clear and easy to answer, and measure what they are supposed to.
What Is a Cognitive Interview?
A cognitive interview is an open-ended qualitative interview in which a researcher asks individual research participants to verbalize their understanding of a set of survey questions. Gallup uses cognitive interviews to gain a better understanding of:
- How easy or difficult it was to understand the survey questions
- Words or phrases that are difficult to understand
- Words or phrases that could be misinterpreted or interpreted inconsistently
- How respondents process answers to survey questions
- Whether respondents interpret words or constructs as the researchers intended
- Whether response categories are mutually exclusive and collectively exhaustive
How Does Gallup Do Them?
Gallup researchers begin the survey development process with a draft questionnaire of the survey items. Once researchers finalize the survey draft, Gallup begins the cognitive interviewing process to test the survey instrument.
For the cognitive interviews, Gallup selects a sample of individuals who are similar to the population that will take the interview based on the final survey instrument. For example, researchers creating a quantitative survey of small-business owners would conduct cognitive interviews with small-business owners. The cognitive interview sample should also be diverse because some subgroups of the population may struggle with specific items or issues more than others.
The purpose of the cognitive interviewing process will be to ensure items are clear, easy to interpret and focused on eliciting the desired information from respondents. In this process, Gallup asks participants to:
- Discuss their interpretation of survey instructions, questions and response choices
- Discuss the process in which they would go about answering the questions
- Provide any recommendations to clarify question wording for unclear or difficult-to-answer items
The cognitive interviewing process allows Gallup to evaluate respondent comprehension, item relevance to the sampled population, the extent to which it is feasible for respondents to answer a survey question and the steps required to select a response.
Using the interview, the researcher can also identify whether and how questions should be refined by leading the interviewee in a guided discussion about these lines of inquiry. Further, the process also allows researchers to identify methods available to reduce total respondent burden and minimize survey fatigue.
Visit the Gallup Methodology Center to learn more about Gallup's rigorous research standards.
