Walsh, Elliot
Hest, Robert
Stewart, Andrea

Survey Data Season Essentials: ACS vs CPS: What Is the Difference Between These Two Federal Surveys?

This post is a part of our Survey Data Season series where we examine data from various surveys that are released annually from the summer through early fall. Find all of the Survey Data Season series posts on our Survey Data Season 2024 page here.

Each year, SHADAC covers the data releases of multiple federal surveys from a variety of agencies, beginning with the National Health Interview Survey (NHIS) in June continuing through the release of Current Population Survey Annual Social and Economic Supplement (CPS ASEC) and American Community Survey (ACS) data products in September through January.

Recently, survey data from both the 2024 Current Population Survey Annual Social and Economic Supplement and the 2023 American Community Survey were released. While these two surveys overlap in a number of topics and similarities, they are also distinct in what they measure and how. 

In this blog, we will discuss the similarities and differences between two of the biggest federal survey resources available: the ACS and the CPS ASEC.

Which Federal Agencies Conduct the CPS and the ACS?

The United States Census Bureau has conducted the Current Population Survey Annual Social and Economic Supplement (CPS ASEC) since 1947 and the current iteration of the American Community Survey (ACS) since 2005 (though Census has conducted similar, prototype ACS surveys since 2000).

The Current Population Survey is jointly sponsored by the Census Bureau and the U.S. Bureau of Labor Statistics (BLS).

When Are CPS and ACS Data Released?

Single-year data from the two surveys is released annually in the second week of September, with supplemental materials for the ACS - such as the Public Use Microdata Sample or “PUMS” files, and the 5-year combined estimates - released later in the year. 

Find the survey data release schedules for these and other survey data resources on our Survey Data Season page. 

Can You Combine or Compare Multiple Years of ACS or CPS Data?

Yes - you can combine and/or compare multiple years of ACS data*. Along with 1-year ACS data files released in September, the Census Bureau has also released 5-year ACS data files annually in December since 2009.

Yes - you can generally combine and/or compare multiple years of CPS data. Care should be taken in combining or comparing across CPS data years, as the survey methodology has been revised numerous times throughout its history. Additionally, because some of the same individuals are surveyed across two CPS data years, researchers should think carefully about how to treat those repeated observations.

On SHADAC’s online data tool, State Health Compare, multiple measures utilize ACS or CPS data that allow users to compare years, trends, and more. We will go over some specific examples of how we use ACS and CPS data later in this blog.

*Except 2020 data, for which the COVID-19 pandemic majorly impacted data collection and distribution efforts. These data are considered “experimental only” and should not be compared to or combined with other years of data. However, the Census Bureau notes that the 5-year files that have the 2020 data year within them are okay for normal use.

Design Differences Between the CPS and the ACS

While both of these surveys are conducted by the Census Bureau, they differ in design, methods, measures, and more. The table below provides an overview of some of the most major differences.

*Geographic level available for data breakdowns - i.e., CPS data is available for the U.S., by region (Midwest, South, Northeast, West) and for states. 

^In-person (ACS & CPS) and mailing activities (ACS) were halted in March 2020. Both resumed in limited capacity in July 2020, and in-person activities resumed fully in September 2020 while mailing activities resumed fully in April 2021.

✝ 2020 data are experimental release only and should not be compared to other years of data.

What Do the ACS vs CPS Measure?

Both the ACS and the CPS gather information on a wide variety of measures.

According to its page on the Census Bureau website, the Current Population Survey, “[provides] information on many of the things that define us as individuals and as a society – our work, our earnings, and our education. In addition to being the primary source of monthly labor force statistics, the CPS is used to collect data for a variety of other studies that keep the nation informed of the economic and social well-being of its people.” 

Specifically, the CPS collects data on measures like:

• Health insurance coverage
• Unemployment
• Labor force participation rate
• Employment data (occupation, industry, class of worker)
• Employment-population ratio
• Child support
• School enrollment
• Demographic data collection (age, race, sex, gender, etc.)

According to its page on the Census Bureau website, the American Community Survey, “helps local officials, community leaders, and businesses understand the changes taking place in their communities. It is the premier source for detailed population and housing information about our nation.”

Specifically, the ACS collects data on measures like:

• Health insurance coverage
• Jobs and occupations
• Educational attainment
• Information on veterans
• Whether people own or rent their homes
• Employment status 
• Disability information
• Housing costs 
• Demographic data collection (age, race, sex, gender, etc.)

You can learn more about the specific questions & measures on the American Community Survey, and why they are asked, on the “Why We Ask Each Question” page on the Census Bureau website.

As you can see, both surveys collect data on similar topics: housing, income, insurance coverage, demographic information, employment, education, and more. A key difference between these two sources is that the ACS provides us with both national and state-level data and estimates for these measures (in addition to lower levels of geography), while the CPS mainly focuses on national-level data.

ACS vs CPS: Guidance on When to Use Each Source 

Knowing what survey data to use depends on what you’re looking at, what measures you’re interested in, the years you’re looking at, and more. The table below guides you through common uses of ACS and CPS survey data, and when to use which source.
 

* While the ACS began in 2005, health insurance coverage questions were not added until 2008.
✝ Except 2020 data, which were “experimental” and should not be compared to other data years.

How Does SHADAC Use ACS and CPS data?

One of the main ways that SHADAC uses the data from these surveys is for our health insurance coverage estimates. These two sources are essential for how SHADAC estimates uninsurance, public insurance, and private insurance coverage rates, including providing information on health insurance coverage nationally, by state, and by demographic categories like race and ethnicity, income, age, and more.

Using the latest 2023 data released in September 2024 from each of these two surveys, SHADAC researchers created two resources explaining overall health insurance coverage estimates. Some of our main findings from these data sources this year include:

• a stable national uninsurance rate for the total U.S. population in 2023 
  • at 7.9% according to the ACS (compared to 8.0% in 2022) 
  • at 8.0% according to the CPS (compared to 7.9% in 2022)
• the rate of uninsurance among children (age 0-18) rose significantly 
  • to 5.8% in 2023 from 5.4% in 2022 according to CPS data
  • to 5.4% in 2023 from 5.1% in 2022 according to ACS data

Read the details on our findings from these two surveys at the links below:

CPS ASEC: 2023 National Health Insurance Coverage Estimates Remain Steady for Adults but Rise for Children

2023 ACS: After Two Years of Decline, Uninsured Rates Increased for Children; Public and Private Coverage Rates See Variations Across the States

SHADAC also uses the wide variety of measures and data available from these surveys on our State Health Compare tool. This online and interactive data tool allows users to create customized data sets and visualizations of state-level health estimates for a number of measures under many categories, including measures on:

• Health Insurance Coverage
• Access to Care
• Cost of Care
• And more

Take a look at the measures available on our State Health Compare (SHC) site that use data from the ACS and CPS!

Measures that Use ACS Data on SHC

Click on any of these measures to explore the data on State Health Compare.

Health Insurance Coverage Type

Percent of adults with fair or poor health status

Percent of households with a broadband internet subscription

Percent of children considered to be poor (<100% FPG)

Income inequality (Gini Coefficient)

Percent of cost-burdened rental households

Measures That Use CPS Data on SHC

Click on any of these measures to explore the data on State Health Compare.

Health Insurance Coverage Type

Percent of people with a high medical cost burden

Median Medical Out-of-Pocket Spending

Percent of adults with fair or poor health status

Have you used State Health Compare to explore data and delve into health care topics? Share your work and tag us on LinkedIn - we love to see how people use SHC to make connections, identify gaps, and work towards making health care accessible & affordable for all people. You can also e-mail us at shadac@umn.edu - we would love to connect!

Stay Updated with SHADAC’s Survey Data Season Series

Stay up to date on our Survey Data Season series, with more Essentials blogs like this one along with other products analyzing newly released data, by signing up for our newsletter and following us on LinkedIn.

WEBINAR: U.S. Census Bureau Data Explained: Breaking Down 2023 Health Insurance Coverage Estimates from the ACS & CPS - featuring a Q&A with a Census Bureau Expert

 

This presentation is a part of our Survey Data Season series where we examine data from various surveys that are released annually from the summer through early fall. Find all of the Survey Data Season series posts on our Survey Data Season 2024 page here.

 

---

On Thursday, September 26th at 1:00 PM CST, SHADAC hosted a webinar covering the release of new Census data on health insurance coverage estimates for 2023. The estimates come from two key federal surveys conducted by the U.S. Census Bureau: The American Community Survey (ACS) and the Current Population Survey (CPS).

SHADAC researchers and presenters Robert Hest and Andrea Stewart discussed the 2023 health insurance data at national and state levels, as well as by coverage type, and a range of other demographic categories (age, geography, poverty level, and more). In addition, SHADAC walked through how to access the data and examples of how to use it to answer research questions. We were also pleased to once again welcome a special guest from the Census Bureau, Sharon Stern, who joined the end of the webinar to participate in a Q&A, answering questions from attendees. 

Webinar attendees came away knowing more about:

• The new 2023 health insurance coverage estimates
• How to access the estimates via Census reports and the data.census.gov website
• How to access state-level estimates from the ACS using SHADAC’s State Health Compare web tool
• When and how to use 2023 Census data to understand health insurance coverage trends

Find a recording of this webinar, along with presentation slides and a transcript of webinar audio, below.

Find the slides from this presentation here. Find a transcript of this webinar here.

Speakers

Elizabeth Lukanen, Moderator
Deputy Director, SHADAC

Ms. Lukanen serves as the Deputy Director for SHADAC, overseeing center research and providing strategic direction and oversight. With 20 years of experience in the health policy arena, Ms. Lukanen specializes in managing complex projects and translating quantitative findings into actionable, policy-relevant information. Ms. Lukanen currently oversees SHADAC’s technical assistance to states in the areas of data use, analysis, and evaluation. Ms. Lukanen is vocal about the importance of collecting and reporting disaggregated data to explore issues related to health equity, while actively working with policymakers to improve data on race, ethnicity, sexual orientation, and gender identity.

 

Robert Hest, Speaker
Senior Research Fellow, SHADAC

Robert Hest joined SHADAC in 2017, providing expertise in survey data, quantitative data analysis, data visualization, and health coverage policy. Mr. Hest leads SHADAC’s work monitoring and investigating developments in the measurement of health insurance coverage and health care access, affordability and utilization in federal survey data. He also manages SHADAC’s State Health Compare website, coordinating data processing, quality assurance, dissemination, and documentation of data used on the site and leads SHADAC’s work in the Minnesota Research Data Center applying restricted-use data to analyze health care affordability, access, and utilization at the state level.

 

 

Andrea Stewart, Speaker
Research Fellow, SHADAC

Andrea Stewart joined SHADAC in 2018 as a Marketing and Communications Specialist, transitioning to a Research Fellow in 2022. Currently, Ms. Stewart leads SHADAC’s efforts to track and report health insurance coverage data from major federal surveys, and is involved in several other ongoing projects, including an effort to explore the potential for the creation of a Medicaid Equity Monitoring Tool. Portions of her other research work range in topic from health equity measurement to understanding the impacts of COVID-19 on federal survey data.

 

 

Sharon Stern, Speaker
Assistant Division Chief, U.S. Census Bureau

Sharon Stern is the Assistant Division Chief for employment characteristics in the U.S. Census Bureau’s Social, Economic and Housing Statistics Division. In her position, Ms. Stern oversees statistics on the labor force, health insurance and disability from several Census Bureau surveys. She has authored a wide variety of Census Bureau reports and papers on topics related to poverty, disability, and health insurance.

 

 

 

Stay up to date with our latest posts by joining our mailing list and following us on LinkedIn.

Tatiana "Annie" Gowlovech
Research Assistant

What Are Statistical Tests? Statistical Testing and How to Determine Statistical Significance — A SHADAC Basics Blog

Basics Blog Introduction

SHADAC has created a series of “Basics Blogs” to familiarize readers with common terms, concepts, and topics that are frequently covered.

This Basics Blog will focus on the topic of statistical tests & testing—including what statistical testing is, how to do a statistical test (specifically, how to do a t test), related definitions, and how to interpret statistical significance, within the context of data drawn from SHADAC’s State Health Compare. 

Keep reading below to learn more about statistical testing and statistical significance.

What Is Statistical Testing? 

Statistical testing can tell us if there is enough evidence to determine if an estimate is statistically different from another estimate, within a certain level of confidence.  

There are various types of statistical tests that are used in different situations. The test used depends on the type of data, variable types, data distribution, and other factors. Some examples of statistical tests include:

• T-Test
• Z-Test
• ANOVA test 

Researchers at SHADAC most often use a “T-Test ” to test for significance in our research and analysis. T-tests are typically used to compare the means or percentages of two independent populations. 

Definitions

To get a better understanding of statistical testing and its meaning in data-driven research, we will walk through an example, using a t-test. Then, we can determine statistical significance from the results of the t-test, which we will explain later in this blog. Before that, however, we will need to define some related terms and values that are used in the t-test formula. 

The following formula is used to conduct t-tests:

Let’s break down this formula and look at the definitions of each of its parts:

Statistical Estimates: approximation of an unknown true population (ex: a sample).

Statistical estimates are the “best guess” of a true population.

Difference in Estimates: the mathematical difference between the statistical estimates (estimate 1 – estimate 2).

In a t-test, estimates are typically a percentage or mean.

Standard Error (SE): estimate of the (un)certainty and precision of a given estimate.

Smaller standard error = less uncertainty, greater precision

Larger standard error = more uncertainty, less precision

T-Critical Value: a value that is the threshold for significance.

T-critical values are set based on what confidence interval is being used. 1.96 is the t-critical value for tests using a 95% confidence interval, which is commonly used.

T-Statistic Value: used to determine whether a value is significant or not

The t-statistic value is compared to the t-critical value to determine significance.

Margin of Error (MOE): within a specified confidence interval expresses that the true population lies within a range of the provided estimate.

The smaller the margin of error, the greater the precision of the estimate.

Confidence Interval: expresses the interval in which we expect the true population value to fall at a given level of certainty.

The greater the confidence interval (ex: 99% compared to 95%), the greater the certainty. For example, we could say we are 95% confident that a value is significant.

How to Do a T-Test: Using State Health Compare

Now, let’s put all of those together. Using State Health Compare, let’s figure out whether the percentage of adults having a telehealth visit in Illinois in 2020-2021 and 2021-2022 are statistically significant from each other.

In 2020-2021, 31.28% of adults had a telehealth visit. In 2021-2022, 27.92% of adults had a telehealth visit.

Figure 1. Percent of Adults in Illinois Who Had a Telehealth Visit by Year

While we can see that a higher percentage of adults had a telehealth visit in 2020-2021 compared to 2021-2022, this doesn’t necessarily mean that the difference is statistically significant.

To find out if these percentages are statistically significant from each other, within a 95% confidence interval, we will conduct a t-test.

First, we will set our t-critical value, which corresponds to what confidence interval you are using. We will be using a 95% confidence interval, which corresponds to a t-critical value of 1.96.

The data from State Health Compare gives us the percentages for each state, as well as the margins of error (MOE). The MOE for 2020-2021 and 2021-2022 are 0.02021 & 0.02175, respectively.

Using the MOE, we can calculate the standard error (SE). To calculate the SE, divide the MOE by our t-critical value, 1.96.

SE for 2020-2021: 0.02021 / 1.96 = 0.01031

SE for 2021-2022: 0.02175 / 1.96 = 0.01109

We will need these numbers to calculate our t-statistic . However, before we can calculate the t-statistic, we need one more value: the difference in estimates.

This value is calculated by subtracting one estimate from the other. We will use the percentages of telehealth visits from each state.

Difference in Estimates of 2020-2021 & 2021-2022: 0.3128 – 0.2792 = 0.0336

Now, we have all the values needed to calculate our t-statistic, which is the ratio of the difference between the two estimates to the variability of the estimates. In other words, it answers: “How close to (or how far away from) ‘0’ is the difference, given the variability between the two estimates?”.

The t-statistic is calculated by plugging in the numbers from above into this formula:

We can now compare this t-statistic to the t-critical value we determined above, which was 1.96. If the absolute value, which is the value of the number regardless of a positive or negative sign, of the t-statistic value is greater than the t-critical value, we can determine that the difference between the estimates is significant. If the absolute value of the t-statistic is less than the t-critical value, we cannot determine there is statistical significance.

Since 2.22 > 1.96, we can determine, with 95% certainty, that there was a significantly higher rate of telehealth visits in 2020-2021 in Illinois than in 2021-2022.  

What This Means & Why It Matters

There are a number of reasons why statistical testing and significance are valuable for researchers and the field of research in general.

Statistical testing is important, as it allows researchers to draw conclusions from the presence or absence of statistical significance, prevents misinterpretation of results, and preserves the integrity of research.  

Statistical significance is important because it tells researchers that the true difference between two population means is greater than zero. In other words, we can say with a specified level of confidence that, as a result of testing, two data points or estimates - or populations, in this particular case of assessing the number of adults who had telehealth visits between years – are different from one another.

However, a finding of statistical significance cannot tell us:

• That the difference is meaningful, just that it is different
• That the comparison makes sense analytically
• That the true difference is equal to the difference we observe 

For more information on statistical testing and an example of how to determine significance when looking at estimates across different states, please read the accompanying brief at this link or by clicking on the image to the right. 

If you’re interested in learning more about statistical analysis, data, and their intersection with health care & insurance coverage, check out some of SHADAC’s other blogs, like:

• Basics Blog: Integrated Health Partnerships (IHPs) — Minnesota’s Medicaid Accountable Care Organization Model
• Survey Data Season 2024: Products and Resources on Latest Data Releases from Federal Survey Sources
• Disability Health Care Data and Information: Resources from SHADAC

Walsh, Elliot
Research Dissemination Coordinator

e wals0421@umn.edu

Disability Health Care Data and Information: Resources from SHADAC

• Unfair treatment in health care settings, at work, or when applying for public benefits
• Adults with disabilities are more likely to live in poverty compared to adults with no disability
• People with a disability often have increased medical expenses, with a study from the National Disability Institute estimating that a U.S. household containing an adult with a disability must spend an estimated 28% more income to obtain the same standard of living as a household with no disability
• Those with disabilities have twice the risk of developing chronic health conditions like depression, diabetes, asthma, and poor oral health

Federal Survey Sample Size Analysis: Disability, Language, and Sexual Orientation and Gender Identity

• People who indicated sexual orientation or gender identity (SOGI)
• People with language access needs, and
• People with disabilities

Collection of Self-Reported Disability Data in Medicaid Applications: A Fifty-State Review of the Current Landscape (SHVS Brief)

States are likely undercounting the number of people with disabilities in their Medicaid programs. While some people with disabilities are covered by Medicaid as a result of disability- related eligibility, little is known about people with disabilities who are eligible on the basis of income or other factors. 

 

This State Health and Value Strategies (SHVS) Brief authored by SHADAC researchers Emily Zylla and Elizabeth Lukanen provides an overview of current disability data collection standards, including information on how states are collecting self-reported disability information on Medicaid applications. SHADAC researchers reviewed paper Medicaid applications for all 50 states and the District of Columbia alongside online applications for 36 states and the District of Columbia. 

 

Understanding and collecting better demographic data about the types of disabilities Medicaid enrollees experience can help us both identify any existing inequities and tailor possible solutions and resources appropriately. Read the brief in full here.

State Health Compare Disability Breakdowns

On SHADAC’s State Health Compare tool, users are able to examine a number of different measures related to health and health care at the state level. A number of these measures can be broken down by demographic categories, including by disability status, including:

• Health Insurance Coverage Type
• Broadband Internet Access
• Adult Unhealthy Days
• Unaffordable Rents

Data disaggregation such as this can help in identifying differences, trends, and similarities between different groups and demographics. See an example of what you can do with this data below, where we breakdown the measure ‘Adult Unhealthy Days’ by Disability Status.

Explore all of the measures including each of their available breakdowns at this link.

Housing Affordability Matters: Unaffordable Rents Infographics Updated with 2022 Data

Using data from the American Community Survey (ACS) in the ‘Unaffordable Rents’ measure on State Health Compare, we created a blog plus six accompanying infographics containing information on the connection between affordable housing and health. Within these publications, we included specific information on housing affordability for households with a person with a disability alongside all rental households to understand any differences in unaffordable rents between demographic groups.

 

For example, in the United States as a whole, 47.3% of rental households had unaffordable rents in 2022. In rental households that include a person that has a disability, that jumps to 54% had unaffordable rents in 2022. 

 

These resources also present data on other demographic groups including Medicaid enrollees, people of color, and household incomes less than 25,000 annually.

 

 

You can find each of the infographics created for the US and five individual states along with an explanatory blog on the topic here.

Minnesota Community and Uninsured Profile

This resource, originally funded by the Blue Cross Blue Shield of Minnesota Foundation, provides users with a tool that provides rates and counts of uninsured Minnesotans at various levels (ZIP code, county, MNSure rating area, state, etc.). The profile then can provide further breakdowns by a number of demographic factors, including by age, income, educational attainment, and disability status.  

 

Explore the profile here.

 

While this profile currently represents communities and uninsured populations from the single state of Minnesota, SHADAC researchers can provide additional technical assistance, profile customization, and analysis to other organizations and groups upon request.

 

If you or your organization is interested in working with this profile, or creating one relevant to you or your needs, don’t hesitate to reach out at shadac@umn.edu or on our contact page here.

Stay Informed on Disability Health Data Resources and Information

The resources and products above represent just some of the work SHADAC is doing that we hope can help advance health equity. Examining different demographic groups, identities, and experiences helps researchers, policymakers, and other stakeholders understand the challenges and barriers different people face – and, hopefully, use that information to tailor health equity work towards those differing needs in the future. 

 

You can stay up to date on our latest releases and updates by following us on LinkedIn, or by signing up for our newsletter.

 

Continue your journey to understand health disparities and health equity with the following products:

• Medicaid Equity Monitoring Tool Project
• LGBT Health Equity: Sexual Orientation and Gender Identity Data Resources from SHADAC
• Provider Discrimination by Sexual Orientation Among Cisgender and Transgender/Nonbinary Adults in Minnesota

Walsh, Elliot
Research Dissemination Coordinator

e wals0421@umn.edu

LGBT Health Equity: Sexual Orientation and Gender Identity Data Resources and Information from SHADAC

• Less likely to have health insurance coverage 
• Less likely to have a regular health care provider
• More likely to delay care
• More likely to report poor quality care and unfair treatment from providers

Sexual Orientation and Gender Identity Data: New and Updated Information on Federal Guidance and Medicaid Data Collection Practices (SHVS Brief) 

State Health Compare: Explore Health Data with SOGI Data Breakdowns

• Adults Who Forgo Needed Medical Care Due to Cost
• Adult Smoking
• Adult Excessive Alcohol Consumption
• Adult E-Cigarette Use
• Chronic Disease Prevalence
• Adult Unhealthy Days
• Activities Limited Due to Health Difficulty
• Adults with No Personal Doctor
• Adult Cancer Screenings
• Adult Flu Vaccinations

Gender Based Discrimination in Health Care by Gender Identity in Minnesota

• Over half (57.1%) of trans and non-binary people reported forgone care—more than double the overall average of 26.2%
• Nearly one-third of trans and non-binary adults had low confidence in getting necessary health care—compared to the overall average of 11.8%

Examining Discrimination and Health Care Access by Sexual Orientation in Minnesota

• Both lesbian/gay and bisexual/pansexual people were more likely to report barriers to health care access
• Bisexual/pansexual people were more likely to report having low confidence in the ability to get needed health care
• Both lesbian/gay or bisexual/pansexual people had significantly higher rates of forgone care

SHADAC Response to 2023 Request for Comments on American Community Survey SOGI Questions

Stay Up to Date on the Latest in SOGI and LGBT Health Data

• Medicaid Equity Monitoring Tool Project
• Affordable Housing and Health: Unaffordable Rents Infographics
• Collection of Self-Reported Disability Data in Medicaid Applications: A Fifty-State Review of the Current Landscape (SHVS Brief)