VISIT STATE HEALTH COMPARE
Blog & News
New SHADAC Brief Looks at Changes in Federal Surveys During COVID Pandemic
August 27, 2021:
The impacts of the coronavirus pandemic have been far ranging—on education, work, healthcare, and almost every other aspect of daily life. However, another effect of COVID that is just now coming to light is the interruption of data collection processes and falling response rates for yearly federal surveys that, among other measures, provide estimates of health insurance coverage for the United States population. Such surveys would, under normal circumstances, be ideally suited to measure changes to insurance coverage during such a tumultuous time.
In a new brief, SHADAC summarizes COVID-era changes and challenges for four major federal surveys—the American Community Survey (ACS), the Current Population Survey (CPS), the Medical Expenditure Panel Survey (MEPS), and the National Health Interview Survey (NHIS)—such as shifting in-person collection methods to phone and email, adding COVID-related questions to questionnaires, extending survey fielding in order to address falling response rates, and delayed data release dates, among others.
All four surveys made significant changes to their Survey Operations:
In-person: The ACS, CPS, and NHIS all suspended in-person operations from March to June 2020, with limited in-person interviews resuming in either July or September 2020. The MEPS Insurance Component (MEPS-IC) suspended in-person interviews entirely.
Mailing: Mailing centers for the ACS and the CPS were shut down in March 2020, but re-opened with limited staffing in July 2020, allowing previously completed surveys to be collected and counted. The MEPS Household Component (MEPS-HC) shifted to dual web-and-mailing collection methods in the fall of 2020 while the MEPS-IC shifted to dual web-and-phone data collection at that time.
Telephone: All surveys (ACS, CPS, MEPS-HC and MEPS-IC, and NHIS) shifted to phone-only surveys while in-person fielding was shut down, or while mailing operations were suspended, in the case of the MEPS-IC.
Web: While both the MEPS-HC and MEPS-IC took a different direction and shifted to telephone-only survey collection in March 2020, the former proposed a shift to dual web-and-mailing data collection modes while the latter opted to shift to dual web-and-phone methods in the fall of 2020.
Three surveys made changes to their Questionnaires:
The ACS and the MEPS-IC opted not make any changes to their questionnaires in response to COVID-19.
The CPS added five new questions regarding COVID’s impact on employment to the questionnaire. The NHIS similarly added new questions to its survey form, but instead asked respondents about COVID testing, contraction and symptoms, and prevention measures.
The MEPS-HC was the only survey to create an entirely new questionnaire: the “Social and Health Experiences” survey, which asked respondents about changes or delays in care due to COVID.
Each survey experienced falling Response Rates:
Where they were able to be measured, survey response rates dropped significantly. Neither the MEPS-HC nor the NHIS has yet released response rates for 2020, but previous years show a drop in response rates for the latter survey of 20 percentage points (80 percent in 2018 to 60 percent in 2019). Both the ACS and CPS saw falling response rates between 2019 and 2020. The ACS dropped 15 percentage points (86 percent in 2019 to 71 percent in 2020), while the CPS fell by roughly 6 percentage points (67.6 percent in 2019 to 61.1 percent in 2020). The MEPS-IC saw a smaller decrease between the years, from 59.6 percent in 2019 to 56.1 percent in 2020.
SHADAC plans to continue monitoring reported changes to these and other federal surveys in 2020, and continuing into 2021 where applicable. As data from 2020 becomes available, our researchers will continue to produce resources to explain what effects these adaptations may have for understanding the data, and if alternative sources may provide ways to fill in gaps where 2020 data may be incomplete or unable to be reliably used.
