Combining Healthcare Quality Measures Into Composites or Summary Scores
The more data you provide on individual measures, the more likely it is that users will be faced with information overload. Combining measures is one way to quickly reduce the number of data points included in your report. This page discusses the use of:
Learn more about Organizing Measures to Reduce Information Overload.
What's a Composite Score?
Composite scores represent small sets of data points that are highly related to one another, both conceptually and statistically. Combining and presenting these items as a single score reduces the potential for information overload.
Types of Composites
- Established by measure developers. In some cases, measure developers use statistical analyses to devise a recommended scoring methodology for combining individual measures into composite scores. This ensures that composites are calculated consistently. For example, all CAHPS surveys produce composite measures that represent a health care organization's performance on two or more related survey questions.
- Improvised. Some report sponsors have combined related clinical measures into a composite score. Because there is no established method for calculating such scores, sponsors have to improvise. For example, Minnesota Community Measurement’s HealthScores website reports a composite score for medical groups that represents the percentage of patients who met recommended treatment goals for diabetes care. While each score could be presented separately, presenting one score makes it easier for consumers to assess overall quality for diabetes care. This approach highlights variation in the scores and communicates that good diabetes care involves multiple elements.
What's a Summary Score?
Summary scores combine many measures into one "overall" score, even though the individual measures may address quite different aspects of quality. While composites include a few measures that are highly related, a summary score reflects many more measures that may address different issues. However, all the measures are about a single specific provider or service.
Example: Consumer Reports. One familiar summary score can be found in Consumer Reports. In addition to detailed ratings, Consumer Reports often combines performance on a number of different characteristics of a product or service into a single overall score. In addition, they often use a checkmark to indicate products or services that are a "best buy." The "best buy" designation captures the relationship between overall quality and price.
Advantages of a Summary Score
- Appeal to the many consumers who want to get to "the bottom line" quickly.
- Greater evaluability.1
- Ease of understanding and use.
- Less space to display (although many reports include both the summary score and the more detailed measures).
Disadvantages of a Summary Score
A summary score is valid if quality is consistent across different services provided by an organization, but all too often that is not the case. For example, many people would like to see an overall score of the clinical quality of hospitals. But most of the time, some units in a hospital perform much better than others. Maternity care may be very high quality in a hospital where cancer care is mediocre. For someone having a baby, the hospital's overall score may not reflect how good the childbirth experience is in that facility; for someone facing surgery for colon cancer, the score may suggest higher quality of care than is actually the case.
A related concern is that overall scores essentially average quality across hospital units, bringing all facilities much closer to being average, with little differentiation. If everyone gets a B–, that reinforces the notion that consumers and patients don't really need to check out quality ratings because there is little variation.
Weighting Elements of a Summary Score
Summary scores must either give the same "weight" to all the measures they include or give some measures more weight than others. Weightings inherently involve judgments of what is more important and consequential. Individual report users may have different views on this than report sponsors, so the summary score may not reflect their preferences.
Sponsors who decide to set weights will need a strong rationale for their decision. Tips for weighting the measures include the following:
- Involve people with multiple perspectives (clinicians, patients, managers, and payers) in setting the weights to make sure they are not biased in the direction of a single group's perspective.
- Make your criteria explicit.
- Explain your process in simple language on the site.
This approach works better if users can look at the individual measures as well as the summary score, since this permits them to do their own (less formal) weighting.
Learn about different perspectives on the use of summary or roll-up measures: Cerully JL, Martino SC, Rybowski L, et al. Using "roll-up" measures in healthcare quality reports: perspectives of report sponsors and national alliances. Am J Manag Care 2017; 23: e202-e207.
Categories as an Alternative to Combining Scores
The use of composite and summary scores can encourage more consumers to use quality information and reduce information overload.
However, when statistically significant correlations do not exist across the measures included in a summary score, these kinds of scores become less defensible. For example, nursing home quality measures derived from the Minimum Data Set have only minimal correlation with each other. As a result, some scores ought not be numerically combined, but they can be organized into categories within a report according to topics.
Learn more about Organizing Measures to Reduce Information Overload.
Reference
- Carman K. Improving quality information in a consumer-driven era: Showing differences is crucial to informed consumer choice (PDF, 670 KB) Presentation delivered at the 10th National CAHPS User Group Meeting. 2006 Mar 31 (PDF, 670 KB). Also see: McGee J. Best Practices for Presenting Quality Data (PDF, 3.77 MB). Presentation delivered at the 11th National CAHPS User Group Meeting, CAHPS College. 2008 Dec 3.
Also in "Generating Scores that Show Differences in Performance"
- Scoring Different Kinds of Measures
- Making Adjustments to Scores
- Combining Measures Into Composites or Summary Scores
- Framing Scores as Positive or Negative
