Hartman, Lacey
Senior Research Fellow

p 612-625-0410
e hartm042@umn.edu

Understanding the Undercount of Medicaid Enrollees in the 2020 Current Population Survey Health Insurance Coverage Data

Studies have long shown that surveys underestimate the number of people enrolled in Medicaid, and that the extent of this “undercount” varies across surveys and states. SHADAC researchers have made significant contributions in this area, and the extent to which the Current Population Survey Annual Social and Economic Supplement (CPS ASEC) undercounts Medicaid is one of the reasons we typically rely on the American Community Survey (ACS) to track state-level health insurance coverage. However, as the COVID-19 pandemic and ensuing disruptions to data collection for this survey resulted in considerable nonresponse bias in the 2020 ACS and led to the decision to release 2020 ACS data in an experimental-only format, SHADAC has chosen to produce 2020 state-level estimates of coverage using the CPS as a stand-in for the ACS for this year only.¹

It is important for users to understand the Medicaid undercount in the CPS when interpreting coverage estimates. In this blog post, we briefly review the research regarding the Medicaid undercount in the CPS, and provide estimates of how it varies across states in 2020. We also discuss the impact of assigning single coverage for those with multiple sources (known as the insurance “hierarchy”) on the Medicaid undercount in the CPS, and other limitations of the data that contribute to the undercount.

Medicaid Undercount
The “Medicaid undercount” refers to the discrepancies that exist between survey estimates of enrollment in Medicaid and the number of enrollees that are actually reported in state and national administrative data—a pattern in which the former estimate is consistently reported lower than the latter. Studies on the subject have shown that nearly all surveys undercount Medicaid/CHIP enrollment relative to administrative sources, but the magnitude of the undercount range can vary broadly between major federal surveys (e.g., the American Community Survey, the Current Population Survey, the Medical Expenditure Panel Survey, etc.).

Previous studies have estimated the Medicaid undercount in the CPS at approximately 30 percent. Research on the Medicaid undercount relies on linking survey data to administrative records to identify whether respondents who were enrolled in Medicaid according to state administrative data reported having Medicaid when surveyed, a process which provides researchers with the best information about the undercount. However, published estimates for 2020 do not exist. Therefore, in order to provide an approximation of the undercount in the 2020 CPS, we compared the weighted count of the population with Medicaid in the CPS to the actual count of Medicaid enrollees based on data from the Centers for Medicare & Medicaid Services (CMS).

Table 1 shows the results of this comparison by state, measuring SHADAC’s estimate of the number of people with Medicaid as a primary source of coverage against the count of Medicaid enrollees from CMS. The undercount varies considerably by state, from just 3 percent in Wyoming to 57 percent in Hawaii.

Reasons for the Medicaid Undercount
Research has shown that the undercount is driven primarily by reporting error; in other words, by people who are enrolled in Medicaid reporting either that they have no insurance or reporting some other type of insurance. As Table 1 demonstrates, the size of the undercount also varies across states. This variation can result from a variety of factors, including how the characteristics of those enrolled in Medicaid differ across states. For example, studies have shown that adults, those with higher incomes, and those with shorter spells of Medicaid coverage are less likely to report having Medicaid when responding to surveys.

There is also reason to consider that the undercount in 2020 may be more severe due to the pandemic. According to recent research, the pandemic resulted in nonresponse bias in the CPS, including the underrepresentation of communities reporting lower incomes, which is likely to mean fewer people reporting Medicaid coverage. It is also likely that the provisions of the public health emergency that provide continuous coverage for those enrolled in Medicaid will result in a larger discrepancy between survey estimates and enrollment, because people that would otherwise become disenrolled due to redetermination of eligibility or for administrative reasons would remain covered.

Impact of Implementing an Insurance “Hierarchy”
Although reporting error is an important driver of the Medicaid undercount in survey data, there are also analytic choices that can impact the extent to which survey respondents are attributed to Medicaid coverage. One of these is the approach of assigning people who report multiple coverage types to just one type of coverage using what is known as an insurance “hierarchy.” SHADAC routinely imposes a primary source of coverage hierarchy when reporting national, state, and sub-state estimates of health insurance coverage. The goal of SHADAC’s coverage hierarchy is to identify survey respondents who report multiple sources of health insurance and determine which source is likely to: (a) be a comprehensive health insurance plan, that (b) serves as the respondent’s primary payer (i.e., the insurance plan that pays first). Despite the analytic soundness of this goal, SHADAC’s hierarchy usually has the effect of increasing the apparent size of the Medicaid undercount.

SHADAC uses separate coverage hierarchies for children (age < 19) and adults (age 19+), as shown in Table 2. The reason for using separate hierarchies by for children and adults is that Medicare is considered the primary source of coverage for adults age 19 and older, as it is the primary payer for covered medical services. For example, if an eligible adult is covered by Medicare either due to age (65+) or disability and is also covered by Medicaid, the primary payer would still be Medicare. Children age 18 and younger are not eligible for Medicare (except in one rare and specific instance), and therefore Employer/Military coverage is considered primary, as many children draw on this source of coverage as dependents of an adult parent or caregiver. Medicaid is considered the payer of last resort in most cases for people who have both Medicaid and some other form of coverage, which is why it is assigned after Medicare and Employer for adults and after Employer coverage for children. More information about SHADAC’s insurance hierarchy is available here.

Table 3 below compares the Medicaid undercount in the CPS with and without application of the SHADAC insurance hierarchy. In the “non-hierarchy” estimates, anyone with Medicaid (regardless of whether they are also covered by another type of insurance) is “counted” as having Medicaid. Not surprisingly, this results in a smaller undercount; in the U.S., the difference between the CPS and CMS data drops from 41 percent to 28 percent. Similar to the undercount estimates with application of the hierarchy, the extent of the change to non-hierarchy data varies across states, from 24 percentage points in Maine to 4 percentage points in Maryland. This variation is due to the difference in the share of people reporting Medicaid and some other coverage type in the CPS. It is also important to note that eliminating the insurance hierarchy does not eliminate the problem of the Medicaid undercount in the CPS overall, or in the vast majority of states.

It is also important to note that the CPS presents unique challenges related both to implementing an insurance hierarchy and the Medicaid undercount. This is primarily due to the reference period of coverage collected in CPS. The CPS gathers information about insurance status over the entire previous year; for instance, if a respondent had separate, non-concurrent spells of Medicaid and employer insurance in the previous year, that respondent would be shown to have both types of coverage in the previous year. With the application of SHADAC’s insurance hierarchy, this person would be assigned to employer-sponsored insurance (ESI) alone, despite having both ESI and Medicaid as primary sources of coverage in the previous year. This is different from the ACS, which captures information about coverage at the time of interview, and therefore all respondents with multiple sources of coverage hold those coverages concurrently.

It is also important to note that there are pros and cons to applying an insurance hierarchy, and the decision to do so or not depends on the research question at hand. A hierarchy may not be appropriate when looking at respondents with a specific source of coverage when the status of that type of coverage as primary versus supplementary is not relevant to the analysis. (e.g., if the question is about understanding the health status of everyone enrolled in Medicaid, regardless of the duration or supplementary status of that coverage). Also, no single hierarchy will be appropriate for every analysis. Rather, analysts can and should alter the priority of coverage types in the hierarchy according to their specific research question. A study focusing on changes in the rates of public coverage over time, for instance, would likely put public coverage types first in the hierarchy before forms of private coverage such as ESI or direct purchase coverage.

Looking Ahead
The CPS has historically undercounted the Medicaid population for a variety of reasons. This issue of undercount does not mean that the CPS is not useful for understanding coverage, but it is important for users to understand this limitation and its causes. SHADAC is available to provide technical assistance to analysts looking to make decisions about how and whether to use the CPS, based on the specific research question of interest.

We are also optimistic that the American Community Survey (ACS) will once again serve as a reliable source of coverage information in 2021, and plan to revert to this data source for providing state-level estimates of coverage. Although Census researchers remain uncertain on whether or not the pandemic may again have impacts on the ACS in the coming year, the fundamental data collection issues (i.e., temporary suspension of mail operations and in-person interviews) that resulted in significant nonresponse bias during the height of the pandemic were not factors for the data collected in 2021.