VISIT STATE HEALTH COMPARE
Blog & News
New Brief Examines Collection and Availability of Data on Race, Ethnicity, and Immigrant Groups in Federal Surveys
September 09, 2021:
Measurement of race, ethnicity, and immigration (REI) status is a critical component of research used to inform policy. However, the standards for measuring these concepts are often confusing for analysts. A new SHADAC brief aims to assist state and federal analysts with survey development and/or analysis of existing survey data to generate estimates of health insurance coverage and access to care across racial and ethnic groups and according to nativity and/or immigrant status.
Survey Background
The new brief presents the collection and classification of survey data for populations defined by race, ethnicity, and nativity/immigrant (REI) status as well as the availability of these data in public use files. We focus in particular on surveys that are conducted by federal agencies and that collect information about health insurance coverage and access to care on an annual or periodic basis for the general population of the United States. Because policy decisions and funding priorities are typically made at the state and local level, the brief emphasizes federal sources that afford state-level estimates as well.
The table below lists the surveys included in the brief, the federal agency responsible for each survey, and whether national and state estimates are available.
| Survey | Federal Agency | National Estimates |
State Estimates |
| American Community Survey (ACS) | Census Bureau | Yes | Yes |
| Behavioral Risk Factor Surveillance System (BRFSS) | Centers for Disease Control and Prevention (CDC) | Yes | Yes |
| Current Population Survey (CPS) | Census Bureau | Yes | Yes |
| Medical Expenditure Panel Survey – Household Component (MEPS) | Agency for Healthcare Research and Quality (AHRQ) | Yes | No |
| National Health Interview Survey (NHIS) | National Center for Health Statistics (NCHS) | Yes | No |
| National Survey of Children’s Health (NSCH) | Census Bureau | Yes | Yes |
| Survey of Income and Program Participation (SIPP) | Census Bureau | Yes | Yes |
While all these surveys must adhere to Office of Management and Budget (OMB) standards for the collection and classification of race and ethnicity in federal surveys, the organizations conducting each survey implement these guidelines differently, resulting in race and ethnicity data that are not always comparable across surveys. In addition, differing design components in these surveys such as target population, sample design and size, and year and frequency of data collection, also account for comparison difficulties.
An Illustration of Measurement Variation: Ethnicity Measures
| Ethnicity Group Categories Collected in Selected Federal Surveys | ||||||||
| Ethnicity Group | ACS | BRFSS | CPS | MEPS | NHIS | NSCH | SIPP | |
| 2019 | 2019 | 2019 | 2018 | 2019 | 2019 | 2018 | ||
| Not Hispanic or Latino | X | X | X | X | X | X | X | |
| Hispanic or Latino | X | X | X | X | X | X | X | |
| Mexican, Mexican-Am., Chicano | X | X | X | X | X | |||
| Mexican | X | X | X | |||||
| Mexican-American | X | X | ||||||
| Chicano | X | X | ||||||
| Puerto Rican | X | X | X | X | X | X | X | |
| Cuban/Cuban-American | X | |||||||
| Cuban | X | X | X | X | X | X | ||
| Cuban-American | X | X | ||||||
| Dominican (Republic) | X* | X* | X | X | ||||
| Central or South American | X | |||||||
| Central American | X | |||||||
| Central American, exc. Salv. | X* | |||||||
| Costa Rican | X* | |||||||
| Guatemalan | X* | |||||||
| Honduran | X* | |||||||
| Nicaraguan | X* | |||||||
| Panamanian | X* | |||||||
| Salvadoran | X* | X* | ||||||
| Other Central American | X* | |||||||
| South American | X* | X | ||||||
| Argentinean | X* | |||||||
| Bolivian | X* | |||||||
| Chilean | X* | |||||||
| Colombian | X* | |||||||
| Ecuadorian | X* | |||||||
| Paraguayan | X* | |||||||
| Peruvian | X* | |||||||
| Uruguayan | X* | |||||||
| Venezuelan | X* | |||||||
| Other South American | X* | |||||||
| Spaniard | X* | |||||||
| Other Latin American | X | |||||||
| Other Hispanic/Latino/Spanish | X | X | X | X | X | X | X | |
| Multiple Categories Allowed | X | X | X | X | ||||
| Other Category Specified | X | X | X | X | ||||
| *Asterisks denote categories that are not specified in survey questions but are reported as open-ended “other” responses. | ||||||||
The differential implementation of OMB standards can be seen in the measurement of ethnicity across the seven surveys included in this brief. In federal surveys, “ethnicity” refers specifically to Hispanic ethnicity. The minimum OMB standard for collecting data on ethnicity includes the options: 1) Hispanic/Latino or 2) Not Hispanic/Latino. All seven surveys ask about Hispanic or Latino origin—and in some cases, Spanish as well—using three main formulations for the ethnicity question: whether the respondent is of Hispanic, Latino, or Spanish origin; what Hispanic/Latino group represents the respondent’s ethnic background; or whether the person considers themselves to be Hispanic or Latino. All seven surveys also include further identifying categories of Hispanic origin as a follow-up question to the primary ethnicity question.
State and Local Level Data
Also included in this new brief are sample sizes for selected REI groups by state for the five surveys that contain state identifiers in their public use files (ACS, BRFSS, CPS, NSCH, and SIPP). However, it is important to note that sample sizes for racial, ethnic, or immigrant groups vary depending on the state or the REI group of interest. Even with publicly available geographic identifiers, the sample sizes for specific REI groups at lower levels of geography may be too small to produce reliable estimates. For example, the BRFSS has a relatively large national 2019 sample of American Indian and Alaskan Native (AIAN) (n=6,639) and Asian (n=8,531) subpopulations. However, there are only 170 AIAN and 287 Asian respondents in the 2019 sample for Minnesota, precluding comprehensive research on health equity for these groups in this state that has a large Asian population and one of the larger American Indian populations.
Download a PDF of the brief for further information.
Related Reading
Collection of Race, Ethnicity, Language (REL) Data in Medicaid Applications
State Health and Value Strategies Brief by SHADAC, 2021
Blog & News
New SHADAC Brief Looks at Changes in Federal Surveys During COVID Pandemic
August 27, 2021:
The impacts of the coronavirus pandemic have been far ranging—on education, work, healthcare, and almost every other aspect of daily life. However, another effect of COVID that is just now coming to light is the interruption of data collection processes and falling response rates for yearly federal surveys that, among other measures, provide estimates of health insurance coverage for the United States population. Such surveys would, under normal circumstances, be ideally suited to measure changes to insurance coverage during such a tumultuous time.
In a new brief, SHADAC summarizes COVID-era changes and challenges for four major federal surveys—the American Community Survey (ACS), the Current Population Survey (CPS), the Medical Expenditure Panel Survey (MEPS), and the National Health Interview Survey (NHIS)—such as shifting in-person collection methods to phone and email, adding COVID-related questions to questionnaires, extending survey fielding in order to address falling response rates, and delayed data release dates, among others.
All four surveys made significant changes to their Survey Operations:
In-person: The ACS, CPS, and NHIS all suspended in-person operations from March to June 2020, with limited in-person interviews resuming in either July or September 2020. The MEPS Insurance Component (MEPS-IC) suspended in-person interviews entirely.
Mailing: Mailing centers for the ACS and the CPS were shut down in March 2020, but re-opened with limited staffing in July 2020, allowing previously completed surveys to be collected and counted. The MEPS Household Component (MEPS-HC) shifted to dual web-and-mailing collection methods in the fall of 2020 while the MEPS-IC shifted to dual web-and-phone data collection at that time.
Telephone: All surveys (ACS, CPS, MEPS-HC and MEPS-IC, and NHIS) shifted to phone-only surveys while in-person fielding was shut down, or while mailing operations were suspended, in the case of the MEPS-IC.
Web: While both the MEPS-HC and MEPS-IC took a different direction and shifted to telephone-only survey collection in March 2020, the former proposed a shift to dual web-and-mailing data collection modes while the latter opted to shift to dual web-and-phone methods in the fall of 2020.
Three surveys made changes to their Questionnaires:
The ACS and the MEPS-IC opted not make any changes to their questionnaires in response to COVID-19.
The CPS added five new questions regarding COVID’s impact on employment to the questionnaire. The NHIS similarly added new questions to its survey form, but instead asked respondents about COVID testing, contraction and symptoms, and prevention measures.
The MEPS-HC was the only survey to create an entirely new questionnaire: the “Social and Health Experiences” survey, which asked respondents about changes or delays in care due to COVID.
Each survey experienced falling Response Rates:
Where they were able to be measured, survey response rates dropped significantly. Neither the MEPS-HC nor the NHIS has yet released response rates for 2020, but previous years show a drop in response rates for the latter survey of 20 percentage points (80 percent in 2018 to 60 percent in 2019). Both the ACS and CPS saw falling response rates between 2019 and 2020. The ACS dropped 15 percentage points (86 percent in 2019 to 71 percent in 2020), while the CPS fell by roughly 6 percentage points (67.6 percent in 2019 to 61.1 percent in 2020). The MEPS-IC saw a smaller decrease between the years, from 59.6 percent in 2019 to 56.1 percent in 2020.
SHADAC plans to continue monitoring reported changes to these and other federal surveys in 2020, and continuing into 2021 where applicable. As data from 2020 becomes available, our researchers will continue to produce resources to explain what effects these adaptations may have for understanding the data, and if alternative sources may provide ways to fill in gaps where 2020 data may be incomplete or unable to be reliably used.
Publication
Changes in Federal Surveys Due to and During COVID-19 Brief
On a recent webinar, the U.S. Census Bureau made the announcement that they will not be releasing 2020 data from the American Community Survey (ACS) as usual in September. Instead, due to the impact of the coronavirus pandemic, “experimental” estimates will be available in November. This substantial change to the data release of a major federal survey highlights one (among many) of the disruptive effects of the coronavirus—the interruption of data collection processes and falling response rates for yearly federal surveys.
Other federal agencies have made their own attempts at adapting surveys to address pandemic-related challenges: delaying data release dates, shifting in-person collection methods to phone and email, adding COVID-related questions to their questionnaire, canceling survey collection for the year, or even creating an entirely new survey, such as the Census Bureau’s new Household Pulse Survey (HPS).
A new brief from SHADAC summarizes these and other COVID-era changes and challenges for four major federal surveys—American Community Survey (ACS), Current Population Survey (CPS), Medical Expenditure Panel Survey (MEPS), and National Health Interview Survey (NHIS)—and explains what effects these adaptations may have for understanding 2020 data as it becomes available.
For more announcements and information related to the 2020 ACS data release changes, visit our webpage.
Blog & News
Census Bureau Announces Major Changes to 2020 American Community Survey (ACS) Data Release
July 30, 2021:*Update (November 30, 2021): On November 30, the U.S. Census Bureau announced the official release of the 2020 ACS 1-year "experimental" estimates, along with the Public Use Microdata Sample (PUMS) file and a working paper.
On July 29, the U.S. Census Bureau announced during a special webinar that due to impacts of the COVID-19 pandemic on the American Community Survey (ACS), the 2020 1-Year estimates will not be released as usual in September, nor will 2020 ACS data be included in related products (e.g., the CPS Health Insurance Report). This change comes as the pandemic resulted in substantially lower response rates and non-response bias that could affect the accuracy of estimates. Instead of the usual data products, the Census Bureau plans to release a set of 2020 ACS “Experimental” 1-year estimates for a more limited set of geographies, scheduled to be available in November 2021.
Resources available from the Census Bureau include:
- Slides from the webinar
- A press release detailing the impacts of COVID on the ACS data
- Revised release schedule and explanation of the experimental estimates
- A short informational guide to understanding how your work may be impacted by the change in the 2020 ACS data
SHADAC will continue to closely monitor future announcements and data releases related to the ACS, and we will assess whether and how any 2020 ACS data products can be used to produce state-level health insurance coverage and related estimates. We will also assess if and which other data sources may be useful for producing the coverage estimates we typically get from the ACS.
Archived Updates
September 22, 2021: The Census Bureau has released national estimates of health insurance coverage from the Current Population Survey (CPS), which they recommend for use this year instead of data from the ACS. SHADAC has published two separate blogs using the CPS, with the first post detailing how the CPS will serve as the primary data source for health insurance coverage this year and how it differs from the ACS and the second blog providing an overview of national-level insurance coverage trends and public and private insurance coverage data from the CPS.
October 6, 2021: Using estimates from the CPS, SHADAC researchers have produced a special, 2020-only Health Insurance Coverage measure for all 50 states on State Health Compare.
October 27, 2021: A new analytic report further detailing the impact of the COVID-19 pandemic on the 2020 ACS 1-year data has been published by the Census Bureau. The report details several different data category responses that confirm initial concerns of over-representation of more economically advantaged respondents.
November 10, 2021: The Census Bureau announced that, in an effort to preserve data quality in the estimates from the ACS, the 2016-2020 5-year data will now be released in March 2022 rather than as initially planned in December 2021.
November 17, 2021: The Census Bureau confirmed that the 2020 ACS 1-year "experimental" estimates will be released on November 30, 2021, along with the Public Use Microdata Sample (PUMS) file and a working paper. Census will also be hosting a webinar accompanying the release in order to explain and answer questions about the estimates.
Blog & News
Measuring Coronavirus Impacts with the Census Bureau's New Household Pulse Survey: Utilizing the Data and Understanding the Methodology
May 22, 2020:The COVID-19 pandemic is having broad-ranging impacts on the health and economic security of Americans across the country, and policymakers, among others, are eager to understand the extent of these impacts and how they vary across states and key subpopulations such as race/ethnicity, given the concerns that historically disadvantaged racial and ethnic minority groups are being disproportionately affected. The Household Pulse Survey (HPS), conducted by the U.S. Census Bureau in collaboration with five other federal agencies, is designed to provide near real-time information on how the COVID-19 pandemic is affecting American households in all 50 states, the District of Columbia (D.C.), and 15 large metropolitan areas. This page hosts the links to SHADAC's ongoing analyses on the take-up of COVID-19 vaccines in the states and provides an overview of the content and methodology of the HPS.
UTILIZATION
COVID-19 Vaccination Rates: New State-level and Subpopulation Evidence from the Household Pulse Survey - Blog Series
As part of SHADAC's commitment to monitoring the impact of the pandemic, we are producing an ongoing series of blogs and data visuals on state vaccination rollout. Recent HPS data provide estimates of COVID-19 vaccination rates at the state level and by subpopulation. This blog series presents top-level findings from these new data, focusing on rates of vaccination (one or more doses) among U.S. adults (age 18 and older) living in a household unit.
Updated post: July 19, 2021
Updated post: April 14, 2021
Updated post: March 25, 2021
Updated post: March 12, 2021
Updated post: February 25, 2021
Updated post: February 11, 2021
Original post: January 29, 2021
METHODOLOGY
Overview
The Household Pulse Survey is designed to provide up-to-date information on the impact of the COVID-19 pandemic on employment, spending, food and housing security, disruptions to education, physical and mental wellbeing, and health insurance and access to care. The survey is being conducted over the course of 12 weeks beginning on April 23, 2020. Invited respondents will be contacted via email and text message and will be asked to take the survey online. Results are scheduled to be published on a weekly basis, one week after the end of each weekly data collection period.
The survey was developed as a collaboration between the U.S. Census Bureau and five additional federal agencies: the USDA Economic Research Service, the Bureau of Labor Statistics (BLS), the National Center for Health Statistics (NCHS), the National Center for Education Statistics (NCES), and the Department of Housing and Urban Development (HUD). The Census Bureau, known for conducting large benchmark surveys like the American Community Survey (ACS), is producing the survey as one of the first projects in its new Experimental Data Products series.
Survey Content
The Household Pulse Survey consists of 50 questions, covering a range of topics designed to measure how households are being impacted by the coronavirus pandemic. These include questions related to health insurance coverage, access to health care, and physical/mental health, in addition to questions concerning income and employment, food security, housing security, and access to education. The content of the questionnaire may be revised after the first one-to-two weeks of circulation in the field based on feedback from expert reviewers and debriefings with field staff.
Health Insurance and Access to Care
The survey contains three questions related to health insurance coverage and access to health care, which are the same health insurance question used by the ACS. The first asks, “Are you currently covered by any of the following types of health insurance or health coverage plans?” Respondents are invited to “Mark Yes or No for each,” allowing them to select from employer-sponsored coverage, direct purchase coverage, Medicare, Medicaid, TRICARE, VA, IHS, and other types of health coverage.
Respondents are then asked two questions aimed at measuring changes in access to care due to the pandemic: “At any time in the last 4 weeks, did you DELAY getting medical care because of the coronavirus pandemic?” and “At any time in the last 4 weeks, did you need medical care for something other than coronavirus, but DID NOT GET IT because of the coronavirus pandemic?” These questions appear to be unique to this survey, as typical questions on this topic ask about care delayed or foregone due to cost or due to other barriers such as transportation, and measures of access to care in other surveys often use the past year as the reference period rather than the last four weeks.
Health Status and Mental Wellbeing
The HPS has one health status question and four questions related to mental health and wellbeing. The survey first asks about current health status: “Would you say your health, in general, is excellent, very good, good, fair, or poor?”
In the next four questions, the survey asks respondents how often in the past seven days (“not at all, several days, more than half the days, or nearly every day”) they have been bothered by each of four problems: “Feeling nervous, anxious, or on edge? Not being able to stop or control worrying? Having little interest or pleasure in doing things? Feeling down, depressed, or hopeless?” The first two questions come from the Generalized Anxiety Disorder 7-Item scale (GAD-7); the second two come from the Patient Health Questionnaire-2 (PHQ-2) screening tool for depression.
Reporting of Estimates
In its initial release, Census published results from the first two weeks of the survey as data tables and via an interactive data tool; the former containing national estimates, and the latter containing a limited set of state-level estimates. Whereas the data tables report only weighted counts and means, the interactive data tool contains a mix of percentages and weighted counts. No standard errors were published in this initial release.
In an effort to quickly report results of the survey, and in contrast with their typical practice, the Census Bureau did not perform any edits or statistical imputation for the majority of the survey results but is instead providing estimates of “Did Not Report” responses as a separate category. This separation makes it difficult to compare count estimates over time and against imputed estimates from other data sources since the count estimates in the HPS are substantially affected by the numbers in the “Did Not Report” category.
In calculating percentages or rates based on these counts, the Census Bureau suggests that excluding the “Did Not Report” category from the denominator will produce percentages that are more comparable to those resulting from statistical imputation. This is the method that the Census Bureau uses to construct the percentages included in the interactive data tool.
Sample Design
The HPS is being conducted for 12 weeks beginning April 23 and utilizes an overlapping weekly panel of respondents, each of whom will be surveyed once per week for three consecutive weeks before being replaced by a new panel. Due to issues with data collection in the first week of the survey, the first contact period was extended one week to cover the period of April 23-May 5, 2020, with the second contact period running from May 7-12. (This post mirrors the Census’ language and refers to these periods as “Week One” and “Week Two.”)
The Census Bureau sampled nearly 1.9 million housing units in Week One of the survey, and more than 1.0 million in Week Two. The Census Bureau plans to sample an additional 1.1 million addresses in each additional week of data collection, with the goal of ensuring that the survey remains representative of the United States’ population. In total, the survey is anticipated to sample approximately 13.8 million housing units and collect roughly 108,000 responses per week, for an approximate 5.0 percent response rate. This was the sample size the Census Bureau deemed necessary to detect a 2.0 percentage-point weekly change in an estimate of 40 percent of the population at national, state, and metropolitan area levels. However, in Weeks One and Two of data collection, the HPS collected only 74,500 and 42,000 responses, respectively, translating to response rates of 3.8 percent and 1.3 percent.
The survey is contacting respondents via email and text message using an address-based sample drawn from the Master Address File. The Census Bureau obtained email addresses and phone numbers from commercial sources, from respondents to the ACS, and decennial census tests, along with other administrative sources. The cell phone frame has more than 87 million addresses with a cell phone number, and the email frame has more than 107 million addresses with an email address. These frames respectively cover 88.0 and 80.0 percent of the country’s addresses. Sampled households are being initially contacted by email if available and then by text message, rotating contact via email and text daily until all contact information has been tried. The survey is being administered online via Qualtrics.
The HPS is designed to be representative of the adult population living in households. Though the survey asks respondents about their household composition (i.e., number of adults and children in the household), the survey only provides information about the primary respondent, rather than all members of the household.
More information about the methodology of the survey, including state-level sample sizes, are available in the Census Bureau’s published technical documentation.
Update January 27, 2021
Phase 3 of the Household Pulse Survey has been extended from its original end date of December 21, 2020 to run through March 1, 2021. Beginning with Week 22 (January 6, 2021 – January 18, 2021), there were slight changes to the questionnaire with the addition of questions regarding individuals’ intention to receive a COVID-19 vaccine and the removal of some questions for which utility has declined over time.
Update November 24, 2020
The HPS has now been extended into three distinct data-collection phases. The third phase is currently underway, having started on October 31, 2020, and will continue through December 21, 2020. The first of the results were published on November 18, 2020, with further results planned for publication every two weeks following that. The second phase of the HPS ran from August 19, 2020, through October 26, 2020 and the data collection time switched from weekly to a two-week period, though for the sake of continuity, each collection period is still called a "week." No changes in either the questionnaire or data collection methods were made between the second and the third phase of the survey.
Update August 20, 2020
The first results from the HPS were published on May 20, 2020 and have continued to be published through July 29, 2020. SHADAC has continued to monitor survey results as they have been released in order to understand how the pandemic is affecting Americans’ health, wellbeing, and access to health care over time. As part of this observation, we recently noted when the Census Bureau proposed an extension of the data collection period for the Household Pulse Survey from 90 days (original end date July 31, 2020) to 24 weeks (end date October 13, 2020). The Office of Management and Budget approved the Census Bureau's request on July 31, 2020. A new version of the survey, complete with a revised questionnaire that focuses on the impact of COVID-19 on education and housing, will be circulated for a second phase of data collection shortly. SHADAC will continue to monitor this second wave of the survey, and will provide updates as results are released.
