Key Questions When Choosing Health Care Quality Measures
When selecting measures to include in your report, you need to consider two issues:
- Whether the measures are "good".
- Whether the measures are appropriate for your audience.
Are the Measures Good?
The easiest way to determine whether a measure is good is to confirm that it has been endorsed by the National Quality Forum (NQF), which checks each measure it endorses against a list of desirable properties.
- Learn more about the NQF’s evaluation criteria.
However, while the NQF has endorsed many measures, you may be interested in measures that have not yet been submitted for endorsement or are in the approval process. You also have to focus on measures that are suitable for public reporting. Whether or not measures are NQF-endorsed, consider assessing for yourself the extent to which the measures have attributes that make them appropriate for reporting purposes.
- Standardization: The measures are standardized at the national level, which means that all health care providers will be reporting the same kind of data in the same way.
- Comparability: If appropriate, the results are adjusted for external factors that could make a health care organization’s performance appear better or worse than it really is; such factors include age, education, gender, income, and health status.
- Availability: Data will be available for the majority of health care organizations that you are profiling.
- Timeliness: The results will be available in time for you to produce and distribute a report when it is most needed by consumers.
- Relevance: The measures address the concerns of your audience.
- Validity: The measures have been adequately tested to ensure that they consistently and accurately reflect the performance of health care organizations.
- Experience: Health care organizations have experience with these measures, so that you can be confident that the measure reflects actual performance and not shortcomings in information systems.
- Stability: The measures are not scheduled to be "retired," e.g., removed from a measurement data set to make room for better measures.
- Evaluability: The results can be evaluated as either better or worse than other results, in contrast to descriptive information that merely shows how health care organizations may be different from each other. For example, a complication rate is an evaluable measure because we know that a lower rate is always better; in contrast, a Caesarian-section rate is not evaluable because we don’t necessarily know whether a higher rate or a lower rate is desirable.
- Distinguishable: The measures reveal significant differences among health care organizations.
- Credibility: The measures are either audited or do not require an audit.
Are the Measures Appropriate for Your Audience?
Once you’ve determined that the measures you are considering are objectively good measures, you next have to decide whether they are appropriate for your purposes. You can start by answering the following three questions.
Does the measure support your goal?
Determining whether the measures are right for your project depends in large part on your project’s goal.
- Improved decision-making: If your goal is to change the behavior of consumers, it is important to select measures that are relevant to consumers, as discussed further in the next question.
- Improved accountability: If your goal is transparency and quality improvement, relevance to consumers may not be quite as important because it is the public nature of the information that motivates providers to improve their performance. The selection of measures for this purpose can also help to educate the public about aspects of care that are important but perhaps not as visible or well-known to them.
Do consumers view the measure as important?
To be useful for consumers, measures of health care quality must capture aspects of health care they view as important, or can easily be shown are important. When consumers do not understand or care about the quality measures used in a report, they are very unlikely to look at the report or use the information in it.
Many existing measures were designed to meet the needs of health care providers and health plans, which use detailed indicators to pinpoint and fix specific problems with the care they deliver. Such measures are usually too narrowly focused and technical to be helpful to consumers.
Unfortunately, we do not know much about which types of measures consumers view as most helpful or least meaningful, and many existing measures have not been “vetted” with consumers. The research that has been done shows that consumers generally respond positively to quality measures that are easy to understand. One implication is that if a measure is described in a way that is clear and understandable, people are more likely to value the measure. Learn more about describing quality measures in consumer reports.
Survey-based measures of patient experience are a good example of measures that are easy to understand. Consumers also like them because they reflect assessments of quality from people similar to themselves.
When given measures described in a way they can understand, consumers also tend to favor process measures that indicate whether the “right thing was done for the patient” over outcome measures. With process measures, the attribution to the provider is clearer to them than with outcome measures.
Is the measure relevant to your intended audience?
A related issue is whether the measure—which may be important to consumers in general—is important to your intended audience. Be sure to match the measures you choose to report to the needs and interests of the intended audience.
For instance, parents of young children will be interested in quality of prenatal care, obstetric outcomes, and childhood immunization rates. Medicare beneficiaries will be interested in quality of care for chronic diseases.
Sometimes it’s not easy to find measures that address a specific target audience. For example, there are more measures addressing women’s health issues (e.g., breast cancer, cervical cancer, osteoporosis, and pregnancy) than those of men.
