Choosing Quality Measures

Choosing the most useful quality measures is one critical step toward success, no matter what your planned uses.

This section:

• Shares some planning suggestions from seasoned users.
• Identifies several factors State users may wish to consider when choosing quality measures.
• Reviews selected measures in terms of those factors.

Online Resource: For more information on uses of quality measures, go to: http://www.ahrq.gov/chtoolbx/uses.htm

Some Planning Suggestions

Circumstances differ across States and regions and over time. Despite such variation, the suggestions in this section have been made by experienced users in a variety of settings and may be useful to consider when developing quality measurement strategies.

Think Long Term

Most uses of quality measurement are multiyear by nature. Discussing quality measurement as a multiyear venture from the beginning helps define realistic expectations and resource requirements.

Collecting quality measurement data for the first time is always a challenge. Therefore, consider designating the first year as a pilot year for which results are not published. This helps avoid publication of results that are more reflective of initial data collection problems than of quality. Publishing first-year results of uncertain quality may well undermine long-term plans.

Consider All Available Measurement Tools

You may want to adopt an entire measurement set. Conversely, you may want to pick and choose among one or a number of measurement sets. Measures in different sets assess differing aspects of child health. Use of selected measures from different sets may provide the more complete or varied assessment you seek.

Think Fewer Measures

Consider using fewer rather than more measures in your first years. This can lay the groundwork for later expansion of the number of measures collected, by demonstrating the feasibility and usefulness of quality measurement activities without overly taxing the resources of all parties involved.

Develop a Plan

The collection, analysis, and use of quality measurement data are demanding, resource-intense activities for State staff as well as health plans and providers. Resources are always stretched. A clear focus on intended uses and information needs will help ensure that you collect only data that will be usable and useful.

Factors to Consider

Many factors are involved in choosing which quality measures to use, whatever the intended purpose. Here are some key considerations:

• Making Comparisons
• Databases and Benchmarks
• Trends Over Time
• Service Delivery Systems
• Unit of Analysis
• Staff Experience and User Support
• Costs

Making Comparisons

Do you plan to use quality measurement results to make comparisons? If so, what are you comparing? Credible comparisons of quality measurement results can be made only where specifications of data elements, data collection, and data analysis procedures are:

• Rigorous.
• Precise.
• The same for all results being compared.

If you want to make broad comparisons across numerous entities or geographic areas, the use of measures that have rigorous and precise specifications is more likely to yield results that can be credibly compared.

Databases and Benchmarks

Do you want to be able to compare your results with an established database of comparable results? Do you want information on how your plan or program ranks with its counterparts? If so, you may want to use measures for which national databases and/or benchmarking information exist. Note that such information may exist only for selected measures in any given measurement set.

Trends Over Time

Do you plan to use quality measurement results to track trends over time? If so, will you track the results of one entity or more? If only one entity is involved, almost any quality measure can be used, provided that a rigorous and highly precise set of specifications is developed and followed with each measure production cycle for:

• Data elements.
• Data collection.
• Data analysis.

If multiple entities are involved, the use of measures with rigorous and precise specifications is more likely to yield results that can be trended simultaneously and used credibly in trend line comparisons.

Service Delivery Systems

Careful consideration of what service delivery systems you have will help steer you to the measures that will produce the best results for:

• Managed care.
• Primary care case management.
• Fee for service.
• All of the above.

Also consider whether you intend to compare results across or within service delivery systems.

Unit of Analysis

Measures are often more suited to one type of unit of analysis than others. Consideration of the following questions will help you choose measures that will produce the best results. Do you plan to collect quality measurement data:

• On a statewide level?
• For managed care organizations?
• For a primary care case management program?
• For a fee-for-service program?

Staff Experience and User Support

Assess the following:

• Does your staff have experience and expertise in quality measurement and data management?
• Is this area a new one for you and your staff?
• Do you anticipate needing extensive, explicit specifications concerning data elements, data collection, and analytical procedures?

Factors such as your own staff resource base or the availability of other local expertise may affect the value of and need for user support. Find out how much user support is offered with various measurement sets and the terms under which it is available. Measurement sets vary in whether they:

• Are backed up by extensive user support resources.
• Include user groups, with varying forms of participation.
• Offer more or fewer types of resources.
• Are free of charge or entail costs.

Costs

Regardless of the level of inhouse expertise, there are unavoidable costs associated with quality measurement. Here are some of the costs you will have to consider:

• Staff resources.
• Senior-level oversight and leadership involvement.
• Staff training (if needed).
• Outside expertise (if needed) .
• For measures not in the public domain, the costs of the measure specifications, software, etc.

• Collection and validation (where needed) of the quality measurement data.

  Examples:

  Survey-based measure—the costs of drawing a sample and administering the survey.
  Nonsurvey instruments—the costs of identifying and verifying administrative data or conducting medical records reviews.

• Analysis of the data and preparation of results.
• Preparation of reports and costs of dissemination, if public dissemination is planned.

When assessing your costs, it is important to remember that quality measurement initiatives may reduce costs through identification of opportunities for cost-effective quality improvement.

Reviewing Some Selected Measures

The information in Factors for Selecting Quality Measures can help determine whether to use one or more of the established quality measures discussed in Established Child Health Care Quality Measures in terms of these factors.

There are other measures that you may wish to consider for use in your program. Emerging Measures provides comments on some child health measurement development work now under way. Mental Health Quality Measures presents current initiatives in the development and use of mental health measures.

Online Resources

For more information on factors for selecting quality measures, go to: http://www.ahrq.gov/chtoolbx/cahptab.htm

For more information on established child health care quality measures, go to: http://www.ahrq.gov/chtoolbx/measures.htm

For more information on emerging measures, go to: http://www.ahrq.gov/chtoolbx/emerging.htm

For more information on mental health quality measures, go to: http://www.ahrq.gov/chtoolbx/measure9.htm