VISIT STATE HEALTH COMPARE
Publication
Significance Testing Using State Health Compare
Originally published December 2016.
Updated March 2023 by SHADAC Research Assistant Adey Fentaw.
This technical brief provides guidance on how to run tests for statistically significant differences using estimates and their associated margins of error from SHADAC’s State Health Compare web tool.
The brief first explains how to conduct a quick visual scan for statistically significant differences using the margins of error (MOE) output in State Health Compare's state ranking charts, and then goes on to provide step-by-step instructions for using margins of error to conduct a more robust test of statistical significance through a type of hypothesis testing known as a “t-test.” Download a PDF of the brief.
Blog & News
Changing Population Estimates: Implications for Data Users
February 21, 2022:.
|
How the CDC’s recent shift will impact calculations for birth and death rates |
The National Center for Health Statistics (NCHS) at the Centers for Disease Control (CDC) recently changed the population estimates they use to calculate population-level rates in their National Vital Statistics System (NVSS), such as birth rates and mortality rates that are commonly accessed through tools such as CDC WONDER. These population estimates allow for the production of rates by providing a population denominator against which to compare the number of births or deaths.
Beginning with the 2021 data year, CDC transitioned from using "bridged-race" population estimates based on the race and ethnicity categories specified in the 1977 OMB Race and Ethnic Standards for Federal Statistics and Administrative Reporting to using "single-race" population estimates from the U.S. Census Bureau (“Census”) based on the race and Hispanic ethnicity categories specified in the 1997 OMB standards for the collection of data on race and ethnicity. The bridged-race estimates had been created to better align the population estimates from Census that used the 1997 OMB categories with the birth and death records from vital statistics systems that in many cases still used the 1977 OMB categories. This transition to the single-race population estimates appears to be precipitated by Census' recent improvement to their collection of race and ethnicity data beginning with Census 2020 data collection and continuing with subsequent years of the American Community Survey.
This move to single-race population estimates has several implications for data users:
- Statistics based on the single-race population estimates are only made available via CDC WONDER for data years 2018 forward, and statistics based on bridged-race population estimates are only available through data year 2020. Data users should not make comparisons between statistics based on bridged-race population estimates and statistics based on single-race population estimates. Further, because of changes made in 2020 to how Census measures race and ethnicity, caution should be used when comparing statistics by race and ethnicity from 2020 and later to data from 2019 and earlier.
- CDC WONDER has expanded the number of race and ethnicity categories available to data users. Previously, only five race categories and two Hispanic ethnicity categories were available. This has been expanded to up to 31 race categories and two Hispanic ethnicity categories.
- The single-race population estimates produced by Census suppress the number of persons less than five years of age at the county level to comply with Census' privacy policies. This prevents users from calculating age-adjusted rates at the county level or by level of urbanization using the single-race data, though crude rates can still be calculated at these levels of geography.
This change also affects SHADAC’s Suicide Deaths measure on State Health Compare. That measure’s Race and Ethnicity breakdown is now available in separate series for data years 1999–2020 and 2018–2021; estimates should not be compared between these series. Furthermore, SHADAC is unable to update the Metropolitan Status breakdown for this measure past data year 2020 due to the previously explained county-level suppressions.
Going forward, data users should keep changes in the denominator in mind when comparing annual estimates based on CDC vital statistics data. SHADAC strives to account for these types of considerations on State Health Compare by making clear when data years aren’t comparable and providing data users with relevant context around changes in methodology and data collection.
Blog & News
Issue Brief: Unwinding the Medicaid Continuous Coverage Requirement - Transitioning to Employer-Sponsored Coverage (State Health & Value Strategies Cross-Post)
January 2023:The following content is cross-posted from State Health and Value Strategies, published on January 19, 2023.
Authors: Elizabeth Lukanen and Robert Hest, SHADAC
Medicaid and the Children’s Health Insurance Program (CHIP) have played a key role in the response to the COVID-19 pandemic, providing a vital source of health coverage for millions of people. The Families First Coronavirus Response Act (FFCRA) implemented a continuous coverage requirement in Medicaid, coupled with an increase in federal payments to states. The requirement has prevented states from disenrolling Medicaid enrollees, except in limited circumstances, allowing millions of Americans continued access to healthcare services during the pandemic.
Enrollment in Medicaid and CHIP has grown sharply since February 2020, with more than 20 million enrollees added to state rosters as of September 2022. Continuous coverage can also likely be credited for the decrease in the number of people who were uninsured in 2021, down to 8.6% from a pre-pandemic level of 9.2% in 2019. This was driven by a 1.4 percentage point increase in public coverage in 2021, to 36.8% from 35.4% in 2019. These trends were mirrored across states, with 28 states experiencing significant decreases in their rates of uninsurance. Meanwhile, 36 states saw rising rates of public coverage with none seeing a decline in public coverage.
When the unwinding of the Medicaid continuous coverage requirement begins, states will restart eligibility redeterminations, and millions of Medicaid enrollees will be at risk of losing their coverage. Estimates vary, but most approximate that in the range of 15 million to 18 million people will lose Medicaid coverage, with some portion exiting because they are no longer eligible, some losing coverage due to administrative challenges despite continued eligibility, and some transitioning to another source of coverage. While much attention has been paid to how states can approach the unwinding of the continuous coverage requirement to prioritize the retention of Medicaid coverage and transitions to marketplace coverage, less attention has been paid to the role of employer-sponsored insurance.
To get a sense for the size of the group that might have employer-sponsored coverage as an option, this issue brief discusses the proportion of individuals with an offer of employer-sponsored coverage by income and state, and the proportion of those offers that are considered affordable based on premium cost. The issue brief also discusses the importance of a Medicaid disenrollment survey to monitor the coverage transitions associated with the unwinding.
A companion issue brief, Helping Consumers Navigate Medicaid, the Marketplace, and Employer Coverage, discusses how state Medicaid agencies, state-based marketplaces, labor departments, and employers can play critical roles in helping people understand and navigate their employer coverage options.
To support communications efforts during the unwinding, SHVS has also produced sample messaging for state departments of labor to share with the employer community which explains the unwinding and coverage options for employees.
Blog & News
Hest, Emily
Lanigan, Lindsey
Navigating Data on State Health Compare (Video)
January 2023:
Check out our new video to learn tips and tricks for using SHADAC’s State Health Compare |
SHADAC’s State Health Compare is a go-to resource for more than 45 data measures on a range of health topics including insurance coverage; cost of care; health behaviors; health outcomes; access to and utilization of care; care quality; public health; and social and economic determinants of health. Estimates are regularly updated from more than 17 data sources and can be broken down across subcategories such as age, education level, race/ethnicity, and other characteristics.
This new video walks through how to create customized data sets and visualizations of state-level health estimates on State Health Compare, allowing analysts, policymakers, and other data users to seamlessly view and download their state’s estimates.
Additional Resources
The SHADAC website is home to numerous blogs detailing ways to use State Health Compare data. Take a deep dive on data related to excessive alcohol use, explore state-level public health funding, observe trends within the opioid epidemic, learn more about physician acceptance of Medicaid patients, or understand rates of unhealthy days for U.S. adults.
Blog & News
Key Resources and 2022 Updates: SHADAC's Health Insurance Unit
October 2022:Updated April 2023
In the years following the passage of the Patient Protection and Affordable Care Act (ACA), SHADAC developed and introduced its Health Insurance Unit, or “HIU,” a tool to aid in defining family interrelationships in federal population surveys for the purpose of analyzing health insurance coverage.
Since the inception of this tool, SHADAC has regularly produced updated HIU variables, including in fall 2022 following the release of the 2021 American Community Survey (ACS) and 2022 Current Population Survey Annual Social and Economic Supplement (CPS ASEC). These variables were recently made available via IPUMS-CPS (October 19, 2022) and IPUMS-USA (March 10, 2023). These HIU variables use the updated HIU methodology introduced in 2020 and incorporate the latest years of survey data and federal poverty guidelines, as relevant.
What is the SHADAC HIU?
First released in 2012, one of the goals of the SHADAC HIU is to establish a common definition of a “family unit” to help ensure consistent and comparative research across major national surveys when analyzing health insurance coverage.
Most federal surveys define a “family” differently from the way it is defined by most private and public insurance programs. For instance, the definition of "family" or "household" in U.S. Census Bureau surveys includes all related members of a household, no matter the degree of relationship, and does not necessarily align with dependent coverage or public program eligibility.
The SHADAC HIU attempts to more closely align with a definition of family used for private and public health insurance coverage eligibility. The SHADAC HIU uses a narrower definition of family that looks at specific interrelationships between individuals within a household and excludes all non-dependent relatives (grandparents, adult siblings, aunts/uncles, etc.) who may be household members but are unlikely to be considered as part of the “family unit” as defined for the purposes of determining eligibility for health insurance.
The SHADAC HIU, then, is defined as an economic unit that consists of those members of a household who would likely be eligible as a group for family health insurance coverage, or whose resources (i.e., income) would be considered in determining eligibility for public coverage.
Related Resources
- Defining Family for Studies of Health Insurance Coverage (August 2021): A brief that outlines the impacts of using the SHADAC HIU in analysis—specifically, analysis showing how the population distribution of family income changes using three different definitions of family: all members in the same household (Census definition), the definition used by the IPUMS (described below), and the SHADAC HIU (described in detail in a companion brief). Researchers can use this brief to assess whether the SHADAC HIU is suitable for their analysis and what the potential impacts of its use might be.
- Stata Code (January 2021): Technical documentation of statistical code in STATA to help researchers to employ the SHADAC HIU in their own analysis using ACS microdata downloaded from IPUMS USA.
