When we launched our World Poll 15 years ago, we knew conducting nationally representative surveys in more than 150 countries wouldn't be easy, even under the best conditions. And over those years, it hasn't been easy, and the conditions haven't always been the best -- sometimes far from it.
But even in the most challenging times -- including natural disasters, revolutions and civil wars -- we've remained steadfast in our commitment to understanding the most pressing problems facing humankind. We've found ways to keep listening to the world when people needed us to be listening the most.
The uncertainty surrounding the COVID-19 pandemic presented us with some unprecedented challenges in carrying out our global surveys. We questioned whether it was even possible to do this safely while maintaining data quality that meets our standards.
However, our mission to keep listening and reporting on the thoughts, feelings and opinions of people worldwide has never been more important. The questions we ask about people's health, wellbeing and food security are the ones that leaders and policymakers desperately need the answers to. So, after careful deliberations and planning, we are pleased to report that we fully relaunched the 2020 World Poll this month.
Going forward, the pandemic has forced us to adapt and modify our approach to how we poll the world. Because the majority of Gallup's World Polls are done face-to-face, we concluded that there was far too much risk to the safety of our interviewers, the people we interview, and the communities they live in, to conduct face-to-face interviews. Instead, Gallup will be conducting its research this year entirely by telephone. We will be reaching more than 100 countries and areas, representing more than 90% of the world's adult population.
Because Gallup has considerable experience in implementing telephone surveys across different regions, we already had a good understanding of what it takes to produce high-quality representative data that are comparable across survey modes. But we still faced a number of challenges.
The original plan for World Poll 2020 involved collecting data for 34 of the 140+ countries by telephone. There will be no change in the methodology in these countries, except many of our in-country partners will be collecting data in a remote working environment. For the rest of the world, we conducted a thorough evaluation to identify countries that could switch to telephone and still produce high-quality data.
Gallup assessed each country or area using the following questions:
-
What portion of the adult population could be represented by landline or mobile phone? Although Gallup's minimum criterion for adequate coverage is 80%, to have sufficient coverage of countries within each region, countries with a minimum of 70% coverage were included in the list.
-
What portion of the adult population could be reached only by mobile phone? To understand issues surrounding representation, we considered the proportion of a country's adult population that uses a mobile phone. We did this because in a number of developing countries, all interviews will be based on randomly selected mobile numbers.
-
What is the overall technological readiness for CATI (computer-assisted telephone interviewing)? Finally, before we could decide whether a country was ready to switch, we reviewed and evaluated each face-to-face country research partner's experience with telephone mode and CATI software and hardware, and country infrastructure (e.g., internet access, electricity) and interviewers' ability to operate in a remote work environment.
As a result of this assessment process, Gallup determined that over 70 current face-to-face countries could switch to telephone mode while adhering to strict standards of survey administration and quality control and complying with social distancing requirements, either in the form of reduced capacity in a call-center environment or remote working.
To increase both coverage and representativeness of specific subpopulations in countries with coverage between 70% and 80%, Gallup is treating mobile phones as a household device instead of a personal device. This will allow all adult household members to be represented and potentially selected for the study.
To reduce the effect of change in mode on data quality, Gallup will recast the current face-to-face questionnaire into a format that is conducive for telephone administration, following the model already in use for other CATI countries. Further, to position its current face-to-face country partners for a successful launch and implementation of telephone surveys, Gallup has:
-
ensured that in-country partners meet technical requirements for optimal telephone administration
-
repurposed its current CAPI (computer-assisted personal interviewing) survey platform for telephone administration
-
designed a comprehensive interviewer training program
Although no one could have dreamed how much the COVID-19 pandemic would disrupt the world, Gallup was well prepared to deploy its contingency World Poll methodology. Because Gallup's World Poll requires the data results to be representative of the adult population in each selected country, Gallup's sampling design, research protocol and quality-control processes all had to be adapted to meet this requirement.
These changes are necessary for us to successfully collect data in 2020, but we also will be studying the potential effects of the mode change in relation to the effects of the pandemic. Gallup is also planning to collect data concurrently in a couple of countries using both CATI and face-to-face methodology, when the timing is appropriate to do so in a safe manner. The results from a concurrent study like this will offer rich insights into the effect that mode has on responses to question items, and will guide the interpretation of the trended results.
For in-country partners in developing countries, continuing the 2020 Gallup World Poll provides not only much-needed work, but also the ability to build research capacity in new areas that otherwise would not have happened, at least not as quickly or within such a compressed timeline.
Over the next several months, Gallup and its partners will be learning a lot in terms of how well the methodological changes are working and how well the teams in each country are adapting to these changes. Look for more stories and updates to follow on news.gallup.com.
Julie Ray contributed to this article.
For complete methodology and specific survey dates, please review Gallup's Country Data Set details.
Learn more about how the Gallup World Poll works.
