Calculating Response Rates for CAHPS Surveys
Response rates are determined by the number of completed surveys (numerator) versus the number of individuals who were eligible to complete the survey (denominator). What counts as a completed survey and the number of eligible individuals depends on which response rate definition is employed.
For CAHPS surveys:
- The numerator is the number of fully completed interviews. (For an explanation of “completeness,” please refer to Tracking Returned CAHPS Questionnaires and Assessing Their Completeness.)
- The denominator uses an estimate of the number of cases of unknown eligibility that are eligible based on available information (rather than all unknown cases).
CAHPS survey response rates are typically calculated with the following formula:
Number of completed surveys
_________________________________________________________________________________________________
Total number of individuals surveyed - (those determined to be deceased + those determined to be ineligible)
Listed below is an explanation of the categories included and excluded in the denominator of the response rate calculation.
Denominator Inclusions
The denominator should include:
- Respondents. The individual returned a questionnaire, whether complete, incomplete, or partially complete.
- Refusals. The individual refused to participate in writing or by phone.
- Nonrespondents. The individual is presumed to be eligible but did not complete the questionnaire for some reason (e.g., never responded, was unavailable at the time of the survey, was ill or incapable, or had a language barrier).
- Bad addresses/phone numbers/emails. The sampled individual is presumed to be eligible even if you are unable to locate them.
Denominator Exclusions
The denominator should not include:
- Deceased. In some cases, a household or family member may inform you of the death of the sampled individual.
- Ineligible. The sampled individual did not receive care or, in the case of a health plan, the sampled individual or child disenrolled from the plan, was never in the plan, or was enrolled in the plan for less than 6 months.
There are other ways to calculate response rates consistent with the guidelines of the American Association for Public Opinion Research (AAPOR). The formula typically used for CAHPS surveys is consistent with the response rate that is reported most often (known as RR3); RR3 includes only fully completed surveys in the numerator. Extensive information about different ways to calculate response rates is available in Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys, AAPOR 2023.
Whichever response rate definition you use, be explicit about the numerator and denominator in your reported response rate so others can evaluate the comparability of other response rates. Based on accumulated knowledge about the response rates from prior surveys, it is possible to gain an accurate understanding of how many individuals should be included in the sample to obtain enough completed surveys.
