Making Adjustments to Health Care Quality Scores

One of the most thorny topics in quality measurement is the adjustment of scores across different plans or providers to account for differences in the characteristics of their patients or themselves. This page reviews key issues related to adjustments to scores.

Adjusting for Risk or Severity

The most common kind of adjustment addresses variations in the characteristics of the patient population, specifically whether—as a group—patients in one facility are at greater risk for poor outcomes. Risk adjustment is a complex statistical enterprise. The extent to which adjustments can be made depends on whether the sponsor has access to other information about the patients or respondents in question, and this in turn typically depends on the source of the data. ^[1][2]

Why adjust for risks or severity?

Sometimes a provider’s patients are more severely ill; sometimes they have other serious co-morbidities. When presenting mortality and other health outcome measures, such adjustments are considered critical for creating a “level playing field” so that a provider is not penalized for taking care of sicker patients.

What do you adjust for?

Most common risk adjustments—for age, prior medical history, or comorbidities—are considered a good idea. It is not considered appropriate, in most circumstances, to adjust for other sociodemographic characteristics such as race, ethnicity, income, education, and/or insurance status. Such adjustments would essentially bury information that could reveal what some would term unacceptable disparities in care.

One alternative is to present information stratified by these categories, if there are enough respondents to generate a reliable score. This has been done when reporting at a national level to policymakers, although some sponsors of reports to the public are also interested in presenting information this way.

EXAMPLE: Presenting Information Stratified by Socio-Demographic Characteristics
 

The Agency for Healthcare Research and Quality’s National Healthcare Disparities Report shows how quality varies across racial/ethnic groups.

Source: Agency for Healthcare Research and Quality. Available at
http://www.ahrq.gov/research/findings/nhqrdr/index.html.

EXAMPLE: Presenting Information Stratified by Socio-Demographic Characteristics
 

The Agency for Healthcare Research and Quality’s National Healthcare Disparities Report shows how quality varies across racial/ethnic groups.

Source: Agency for Healthcare Research and Quality. Available at http://www.ahrq.gov/research/findings/nhqrdr/index.html. Accessed October 14, 2022.

EXAMPLE: Presenting Information Stratified by Socio-Demographic Characteristics
 

The Centers for Medicare & Medicaid Services has several stratified reports measuring and reporting on the nature and extent of health care disparities.

Source: Centers for Medicare & Medicaid Services. Available at https://www.cms.gov/About-CMS/Agency-Information/OMH/research-and-data/statistics-and-data/stratified-reporting. Accessed October 14, 2022

Adjusting for Provider Size

Sometimes the providers you are comparing differ widely in size. This can lead to a situation in which the number of observations for a given measure can also vary, with a large “n” for a larger organization and a smaller “n” for a smaller organization. In this case, normal comparisons of scores can be unreliable; there is a danger that the performance of smaller organizations will look artificially lower than that of larger organizations.

Many statisticians believe that under these circumstances, it is appropriate to adjust scores using a method called hierarchical modeling or smoothing. Since the statistics involved are quite complex, it is critical to find an expert who can both conduct the analysis and explain what was done in reasonably nontechnical terms to the individuals and organizations with whom you are working.^[3]