Asking Inclusive Questions About Gender: Phase 1

WASHINGTON, D.C. -- Gender is an important piece of demographic information that Gallup collects in its polls worldwide. Attitudes and experiences often differ in important ways by gender, and collecting this information is crucial to understanding people’s experiences.

Gallup has historically collected gender information in two categories: male and female. However, respondents who identify as nonbinary, transgender or another identity that does not conform to the male/female gender binary are not represented in these categories. This presents a challenge because it is important to offer respondents survey questions and response options that reflect their experiences.

Gender is also commonly used in post-stratification weighting adjustments and must be asked in a way that allows for calibration to weighting targets generated from government statistical systems (such as the U.S. Census). The 2020 Census asked, “What is your sex?” with the response options of male and female. Gender (collected in most Gallup surveys) and sex (asked on the U.S. Census) are often used interchangeably during weighting, which has historically presented little issue. But sex and gender are more complex than this.

In 2022, Gallup reported that 0.6% of adults identify as transgender, but this number differs significantly by generation. Less than 0.05% of Silent Generation adults identify as transgender, compared with 1.9% of adults from Generation Z. Generational changes in gender identity may fundamentally change how sex and gender are measured and how datasets are weighted to accurately reflect sex and gender.

To address these changes, Gallup began testing new gender questions. We reviewed the literature to explore how other surveys and statistical systems are collecting gender data. We used what we learned to construct our own question versions and conducted several rounds of testing.

We wrote the questions in the first round of testing with the following specific objectives in mind:

• The questions should be appropriate for global use, including in countries where it is illegal and potentially life threatening to outwardly identify as transgender or gender nonconforming.
• The question and response options should use terminology that is familiar and can broadly capture different gender identities. We understood that not everyone would agree on the best terminology and that terminology continues to evolve, but we wanted it to be as broadly inclusive as possible.
• Gender identity can be a divisive issue in the United States and many other countries. A version of the question should be used that does not increase item nonresponse or breakoffs from respondents who may object to a more inclusive gender identity question.
• The question should minimize issues with calibrating to weighting targets.

We tested three different versions of the question, in a web survey experiment of 36,131 U.S. adults, conducted Sept. 1-17, 2021, via the Gallup Panel. The question text was the same in all three versions; only the response options varied.

Version 1

In the first version of the question, we added “Prefer not to say” to the "Male" and "Female" response options. The “Prefer not to say” response can serve many purposes. It can be interpreted as a refusal, which is an option if respondents are uncertain about how to categorize their gender, or it can be used as an option for people who do not identify as male or female.

What is your gender?

1. Male
2. Female
3. Prefer not to say

Version 2

In the second version, we added “Nonbinary” and “Gender not listed here” to the response options. We believed “Nonbinary” would be widely understood and accepted terminology. However, we added “Gender not listed here” to see how many people (who did not select male or female) would select something other than nonbinary. We intentionally chose to use “Gender not listed here” versus another phrasing, such as “Other,” which can convey “otherness.” However, we did not provide a text box so respondents could specify their gender.

What is your gender?

1. Male
2. Female
3. Nonbinary
4. Gender not listed here (no text box provided for fill-in responses)
5. Prefer not to say

Version 3

In the third version, we removed “Nonbinary” and added a text box after “Gender not listed here” to see what terms were most commonly used, unprompted.

What is your gender?

1. Male
2. Female
3. Gender not listed here (open-ended text box provided for fill-in response)
4. Prefer not to say

Results

In each of the question versions, approximately 1.5% of respondents provided an answer other than male or female. In version 1, 1.4% of respondents selected “Prefer not to say,” compared with approximately 1.0% in version 2 who said the same (and in which respondents were able to select “Nonbinary” or “Gender not listed here”) and 1.2% in version 3 (which also had “Gender not listed here” but gave respondents the option of specifying their gender).

This seems to indicate that in version 1, respondents who do not identify as “Male” or “Female” may have used the “Prefer not to say” option. This question version could be useful for research topics or settings where gender is a sensitive question, such as in countries that criminalize people who are trans.

In version 2, 0.4% selected “Nonbinary,” and 0.1% selected “Gender not listed here,” compared with 0.4% who selected “Gender not listed here” in version 3.

In version 3, people who said, “Gender not listed here” were asked to type their gender. About 17% of the small number of write-in responses were a critique of the question or an invalid response and included answers such as “Only two genders” or “Master of the universe.” Another 10% wrote in either “Male” or “Female.” Of the valid answers provided, the most common response was "Nonbinary" (n=14), followed by "Agender" (n=3), "Gender-fluid" (n=3), "Trans/Transgender" (n=3), "Genderqueer" (n=1), "Noncis" (n=1), "Two-spirit" (n=1), "Demigirl" (n=1) and other answers that couldn’t be coded (n=3).

The combined results of versions 2 and 3 seem to indicate that “Nonbinary” is the most broadly accepted term, if only one term is used in the question. For most research, we do not recommend adopting an open-ended response option. Many of the responses were a critique of the question versus a valid response option, and coding open-ended response options can be a burden on resources.

Conclusions

Based on the results of this experiment, Gallup is most commonly using the following question for U.S.-based public opinion polling of adults:

What is your gender?

1. Male
2. Female
3. Nonbinary

For much of this research, the primary objective is to provide respondents with inclusive response options. We recognize that people have diverse identities and experiences, and people who do not identify as a man or woman cannot be treated as a homogenous group. For most public polling projects, sample sizes are not large enough to report the diversity of gender experiences. For these types of studies, a broad category such as nonbinary may be suitable.

Some researchers are concerned that more inclusive response options may turn off respondents who reject the idea that there are more than two genders. Although there were some critical responses to the open-ended question, 99.9% of respondents answered the question without incident.

The “Prefer not to answer” option was used in all the question versions. There is evidence that this option could prove useful in countries or settings where gender is a highly sensitive topic. However, our experiment does not give us clear information about the various motivations for selecting this response. Some individuals may have selected it when they could not find an appropriate category, while others may have used it as the equivalent of a refusal.

Gallup is continuing to research how gender and gender experiences are measured on survey questions. The second blog about this research will be shared in summer 2023 and will explore questions that can be used for accurately capturing or screening the diversity of gender experiences.

To stay up to date with the latest Gallup News insights and updates, follow us on Twitter.