As part of SHADAC's commitment to monitoring the impact of the pandemic, we are producing an ongoing series of blogs and data visuals on state vaccination rollout. Recent HPS data provide estimates of COVID-19 vaccination rates at the state level and by subpopulation. This blog series presents top-level findings from these new data, focusing on rates of vaccination (one or more doses) among U.S. adults (age 18 and older) living in a household unit.

Updated post: July 19, 2021

 Updated post: April 14, 2021
 Updated post: March 25, 2021
 Updated post: March 12, 2021
 Updated post: February 25, 2021
 Updated post: February 11, 2021
 Original post: January 29, 2021

METHODOLOGY

Overview
The Household Pulse Survey is designed to provide up-to-date information on the impact of the COVID-19 pandemic on employment, spending, food and housing security, disruptions to education, physical and mental wellbeing, and health insurance and access to care. The survey is being conducted over the course of 12 weeks beginning on April 23, 2020. Invited respondents will be contacted via email and text message and will be asked to take the survey online. Results are scheduled to be published on a weekly basis, one week after the end of each weekly data collection period.

The survey was developed as a collaboration between the U.S. Census Bureau and five additional federal agencies: the USDA Economic Research Service, the Bureau of Labor Statistics (BLS), the National Center for Health Statistics (NCHS), the National Center for Education Statistics (NCES), and the Department of Housing and Urban Development (HUD). The Census Bureau, known for conducting large benchmark surveys like the American Community Survey (ACS), is producing the survey as one of the first projects in its new Experimental Data Products series.

Survey Content
The Household Pulse Survey consists of 50 questions, covering a range of topics designed to measure how households are being impacted by the coronavirus pandemic. These include questions related to health insurance coverage, access to health care, and physical/mental health, in addition to questions concerning income and employment, food security, housing security, and access to education. The content of the questionnaire may be revised after the first one-to-two weeks of circulation in the field based on feedback from expert reviewers and debriefings with field staff.

Health Insurance and Access to Care
The survey contains three questions related to health insurance coverage and access to health care, which are the same health insurance question used by the ACS. The first asks, “Are you currently covered by any of the following types of health insurance or health coverage plans?” Respondents are invited to “Mark Yes or No for each,” allowing them to select from employer-sponsored coverage, direct purchase coverage, Medicare, Medicaid, TRICARE, VA, IHS, and other types of health coverage.

Respondents are then asked two questions aimed at measuring changes in access to care due to the pandemic: “At any time in the last 4 weeks, did you DELAY getting medical care because of the coronavirus pandemic?” and “At any time in the last 4 weeks, did you need medical care for something other than coronavirus, but DID NOT GET IT because of the coronavirus pandemic?” These questions appear to be unique to this survey, as typical questions on this topic ask about care delayed or foregone due to cost or due to other barriers such as transportation, and measures of access to care in other surveys often use the past year as the reference period rather than the last four weeks.

Health Status and Mental Wellbeing
The HPS has one health status question and four questions related to mental health and wellbeing. The survey first asks about current health status: “Would you say your health, in general, is excellent, very good, good, fair, or poor?”

In the next four questions, the survey asks respondents how often in the past seven days (“not at all, several days, more than half the days, or nearly every day”) they have been bothered by each of four problems: “Feeling nervous, anxious, or on edge? Not being able to stop or control worrying? Having little interest or pleasure in doing things? Feeling down, depressed, or hopeless?” The first two questions come from the Generalized Anxiety Disorder 7-Item scale (GAD-7); the second two come from the Patient Health Questionnaire-2 (PHQ-2) screening tool for depression.

Reporting of Estimates
In its initial release, Census published results from the first two weeks of the survey as data tables and via an interactive data tool; the former containing national estimates, and the latter containing a limited set of state-level estimates. Whereas the data tables report only weighted counts and means, the interactive data tool contains a mix of percentages and weighted counts. No standard errors were published in this initial release.

In an effort to quickly report results of the survey, and in contrast with their typical practice, the Census Bureau did not perform any edits or statistical imputation for the majority of the survey results but is instead providing estimates of “Did Not Report” responses as a separate category. This separation makes it difficult to compare count estimates over time and against imputed estimates from other data sources since the count estimates in the HPS are substantially affected by the numbers in the “Did Not Report” category.

In calculating percentages or rates based on these counts, the Census Bureau suggests that excluding the “Did Not Report” category from the denominator will produce percentages that are more comparable to those resulting from statistical imputation. This is the method that the Census Bureau uses to construct the percentages included in the interactive data tool.

Sample Design
The HPS is being conducted for 12 weeks beginning April 23 and utilizes an overlapping weekly panel of respondents, each of whom will be surveyed once per week for three consecutive weeks before being replaced by a new panel. Due to issues with data collection in the first week of the survey, the first contact period was extended one week to cover the period of April 23-May 5, 2020, with the second contact period running from May 7-12. (This post mirrors the Census’ language and refers to these periods as “Week One” and “Week Two.”)

The Census Bureau sampled nearly 1.9 million housing units in Week One of the survey, and more than 1.0 million in Week Two. The Census Bureau plans to sample an additional 1.1 million addresses in each additional week of data collection, with the goal of ensuring that the survey remains representative of the United States’ population. In total, the survey is anticipated to sample approximately 13.8 million housing units and collect roughly 108,000 responses per week, for an approximate 5.0 percent response rate. This was the sample size the Census Bureau deemed necessary to detect a 2.0 percentage-point weekly change in an estimate of 40 percent of the population at national, state, and metropolitan area levels. However, in Weeks One and Two of data collection, the HPS collected only 74,500 and 42,000 responses, respectively, translating to response rates of 3.8 percent and 1.3 percent.

The survey is contacting respondents via email and text message using an address-based sample drawn from the Master Address File. The Census Bureau obtained email addresses and phone numbers from commercial sources, from respondents to the ACS, and decennial census tests, along with other administrative sources. The cell phone frame has more than 87 million addresses with a cell phone number, and the email frame has more than 107 million addresses with an email address. These frames respectively cover 88.0 and 80.0 percent of the country’s addresses. Sampled households are being initially contacted by email if available and then by text message, rotating contact via email and text daily until all contact information has been tried. The survey is being administered online via Qualtrics.

The HPS is designed to be representative of the adult population living in households. Though the survey asks respondents about their household composition (i.e., number of adults and children in the household), the survey only provides information about the primary respondent, rather than all members of the household.

More information about the methodology of the survey, including state-level sample sizes, are available in the Census Bureau’s published technical documentation.

Update January 27, 2021
Phase 3 of the Household Pulse Survey has been extended from its original end date of December 21, 2020 to run through March 1, 2021. Beginning with Week 22 (January 6, 2021 – January 18, 2021), there were slight changes to the questionnaire with the addition of questions regarding individuals’ intention to receive a COVID-19 vaccine and the removal of some questions for which utility has declined over time.

Update November 24, 2020
The HPS has now been extended into three distinct data-collection phases. The third phase is currently underway, having started on October 31, 2020, and will continue through December 21, 2020. The first of the results were published on November 18, 2020, with further results planned for publication every two weeks following that. The second phase of the HPS ran from August 19, 2020, through October 26, 2020 and the data collection time switched from weekly to a two-week period, though for the sake of continuity, each collection period is still called a "week." No changes in either the questionnaire or data collection methods were made between the second and the third phase of the survey.

Update August 20, 2020
The first results from the HPS were published on May 20, 2020 and have continued to be published through July 29, 2020. SHADAC has continued to monitor survey results as they have been released in order to understand how the pandemic is affecting Americans’ health, wellbeing, and access to health care over time. As part of this observation, we recently noted when the Census Bureau proposed an extension of the data collection period for the Household Pulse Survey from 90 days (original end date July 31, 2020) to 24 weeks (end date October 13, 2020). The Office of Management and Budget approved the Census Bureau's request on July 31, 2020. A new version of the survey, complete with a revised questionnaire that focuses on the impact of COVID-19 on education and housing, will be circulated for a second phase of data collection shortly. SHADAC will continue to monitor this second wave of the survey, and will provide updates as results are released.