All-Payer Claims Databases Measurement of Care: Systematic Review and Environmental Scan of Current Practices and Evidence

State-and national-level momentum is building to create and maintain all-payer claims databases (APCDs), with increasing interest from multiple stakeholders in improving the value of health care in order to achieve the Triple Aimⁱ of better health, better quality, and more efficiency. APCDs are large-scale databases that systematically collect professional, facility, pharmacy, laboratory, and dental claims (typically, but not always), as well as eligibility and provider files from private and public payers.¹

Some States have also established consumer-facing Web sites that support price transparency for consumers. Interest is growing in how to leverage APCD data for consumer-facing Web sites for price transparency, as well as for efforts to understand new innovations in care, such as accountable care organizations.

Return to Contents

Objective

The objective of this literature review and environmental scan is to map an approach to creating an inventory of measures of quality, cost, and utilization of care across settings for potential use with an APCD, noting gaps or current barriers to APCD measurement. The literature review and environmental scan provide a foundation for the work, describing the breadth of available measures and generating a framework for choosing measures and organizing the final inventory. This effort was funded by the Agency for Healthcare Research and Quality (AHRQ).

A large number of measures are based on administrative health data, in particular, hospital discharge abstract databases. For this report, the focus was on measures that leverage the unique aspects of APCD data, including longitudinal data from multiple sources that allow patients to be tracked across time and settings, pharmacy data, and data on dollar amounts paid by insurers and patients. These key characteristics of APCD data enable measurement not possible with hospital discharge claims data alone, in particular, measures of ambulatory care (including measures that require data from multiple settings), episode of care measures, and measures of cost.

Return to Contents

Guiding Questions

Several guiding questions were used to focus the search strategy and data collection. The guiding questions, which speak to the objectives of the report, were:

1. What measures or outcomes (quality, utilization, safety, price, etc.) that leverage the unique data in APCDs have been reported in the scholarly literature or in online public reports using APCD data?
2. What measures or outcomes have been proposed for use with APCD data or claims data that are episode based or longitudinal in nature?
3. What important measure gaps have been noted in relation to transparency initiatives? Have APCD-specific measure concepts been proposed to fill these gaps (even if no fully specified measures yet exist)?
4. What potential barriers to using and reporting measures with APCD data have been identified in the peer-reviewed or grey literature, including issues around availability and access to data elements? What strategies for overcoming these barriers have been proposed in the literature?
5. What are some of the methodological considerations pertaining to using APCD data for measurement that have been discussed in the peer-reviewed or grey literature?

Return to Contents

Key Findings From Literature Review and Environmental Scan

The peer-reviewed literature search yielded two basic types of articles: overview papers presenting concepts related to measurement in our areas of focus and those presenting particular measures used to address the authors’ specific research question. A total of 204 articles were screened; of those, 98 articles were included in the full review, including 17 overview articles and 81 research articles.

Overview articles presented concepts related to measurement using APCDs or other large claims databases. The research articles focused on measurement of cost, quality, or utilization using APCDs, multi-payer claims databases (MPCDs), or other claims databases (Medicare or Medicaid). The measures found most often focused on a specific research question or discussed the application of specific measures for public reporting or price transparency initiatives.

The environmental scan yielded information on existing APCDs and the potential for using APCDs to measure quality, cost, and utilization of care across populations and settings. In total, 236 different sources of information were identified using initial search criteria; after preliminary review, 127 sources that met our inclusion criteria were included in this report.

Sources found in the environmental scan came from AHRQ expert materials, the APCD Council, reports, task force papers, policy briefs, trade papers, business journals, white papers, books, APCD public reporting Web sites, measure inventories such as the National Quality Forum, and other sources. National and State-specific general resources most often described the basis for the national trend to develop APCDs as well as State-specific issues related to building or implementing APCDs.

State resources, in particular, focused on building the case for APCDs and describing the barriers to establishing APCDs and using them for measurement. Many States with active APCDs have also issued reports of statewide quality, utilization, and cost.

Finally, resources related to price transparency were also included, as this is a major application of APCDs. These resources included high-level summaries of current efforts and barriers from governmental and nongovernmental sources, as well as State-level price transparency reports.

Major Organizations and Key Contributors to APCDs and Measurement

Several national organizations have developed expertise with APCDs or measurement using APCDs and have created resources and provided support for States creating APCDs. Such organizations include AHRQ, which leads and supports several initiatives:

• United States Health Information Knowledge (USHIK) database, a repository for State APCD file submission specifications and data elements.
• Community Quality Collaboratives, community-based organizations of multiple stakeholders (e.g., health care providers, purchasers, health plans, consumer advocacy organizations) working together to transform health care at the local level.
• National Quality Measures Clearinghouse (NQMC), an online, searchable inventory of evidence-based measures and measure sets.

Nongovernmental organizations have also played a key role in advancing the science and implementation of APCDs:

• The APCD Council and the National Association of Health Data Organizations (NAHDO) have taken the lead in supporting and documenting current State efforts and legislative work around APCDs. These efforts include working to harmonize data collection and release across States and providing technical and policy support to States that have or are developing APCDs.
• Catalyst for Payment Reform is a nonprofit organization that brings the perspective of purchasers to APCD efforts. They also provide tools such as report cards on States’ efforts on price transparency to help purchasers and other stakeholders understand issues related to payment reform and transparency.
• The Health Care Cost Institute is a nonprofit organization whose goal is to provide access to health care cost and utilization data to researchers and policymakers trying to understand the factors influencing health care costs. They create twice-yearly cost reports based on claims from four major insurers.
• The Robert Wood Johnson Foundation (RWJF) has also been instrumental in bringing together multiple stakeholders to improve the quality of health care. For example, their Aligning Forces for Quality initiative has resulted in public report cards about quality in some States.
• The National Quality Forum (NQF) is a leader in endorsing and encouraging implementation of evidence-based, valid, reliable measures that are meaningful to stakeholders, including consumers.

Other major sources of measures for this report included the literature review, public reporting Web sites from APCDs and other online reports of cost and quality, and individual organizations such as the National Committee for Quality Assurance (NCQA), the Quality Alliance Steering Committee (QASC), and Bridges to Excellence.

Major Sources of Measures

The table below gives a high-level overview of the key sources of measures identified through the environmental scan and literature review. The focus was on measures that leverage the strengths of APCDs, namely, ambulatory measures, episode of care measures, and cost measures. This review formed the basis of a measure inventory for use with APCDs; the first phase of measure inventory development entailed prioritizing measure sources, types, and focus, after which details of individual measures were compiled for further review.

Table ES-1. Number of Measures or Public Reports for Potential Use With APCDs

Key: NQF: National Quality Forum; APCD: all-payer claims database; NCQA: National Committee on Quality Assurance.
* These papers provide measures or potential measures by describing one measure specific to the study question, using claims data, or describing the use of a group of measures that are already in use and are described elsewhere (e.g., NQF-endorsed measures, Centers for Medicare & Medicaid Services measures).
^† From a list compiled in Evidence-based Practice Center Technical Brief Protocol. Public Reporting of Cost Measures in Health.²

In this report, a variety of measure sources for use with APCDs are reviewed in greater depth. The literature provides studies using individual measures, some with well-described technical specifications. The NQF and NQMC are measure aggregators and provide access to structured technical specifications. Public reporting Web sites either explicitly use State APCD data or use a combination of several data sources and measures. Some of these can be adapted for use with APCD data, but technical specifications are not always easily available through the online resource. Lastly, several organizations are contributing discrete groups of claims-based measures, such as NCQA, QASC, and Bridges to Excellence, who have made the technical specifications publicly available or available on request.

Key Categories and Domains for Measures That Leverage APCD Data

The project team identified key concepts and measure categories described in the literature review and environmental scan. These key categories and domains can be used to organize the measure inventory. They can also be used to prioritize certain categories of measures. For instance, the purpose of performance reporting (e.g., choice, negotiation, accountability) can help guide measurement choice in the following way: if the purpose of a public report is to support consumer choice of providers, then a cost measure that only shows the average insurance reimbursement rate without including the patient out-of-pocket cost for each provider will not be helpful. However, if the purpose of the measure is to assist in negotiations between insurers and providers, average insurance reimbursement could be more useful.

In addition, measures may be used for population health and policy purposes. For example, population-level measures of utilization and cost are important to strategic planning to help eliminate health care disparities at the State, regional, and local levels. APCD data may also be used in State operations such as budgeting and rate review. In the box below, we list some major categories to consider in creating the measure inventory.

The team explored options for including a categorization related to the audience for measure reporting—consumer, provider, payer, or policymaker—based on feedback received from the Technical Expert Panel during the February 2015 meeting.

Barriers to Using APCDs for Measurement and Potential Solutions

Although APCDs are meant to contain comprehensive claims data across settings and time, there are still many barriers to using APCDs for measurement. Issues with data completeness, quality, standardization, and access hamper such efforts. Identifying and resolving these barriers is critical to the ability to use APCDs to improve health care value. The table below summarizes key barriers related to APCD data collection and use and potential solutions.

Table ES-2. Key Barriers and Potential Solutions Related to Data Availability, Quality, and Access

Methodological Issues or Barriers Pertaining to Using APCD Data for Measurement

As more States develop APCDs and public reporting Web sites based on APCD data, it is critical to find valid measures relevant to stakeholders and feasible to implement on a large scale. Multiple methodological issues have arisen in pursuing this goal. Key issues discussed in the report include:

• Inadequate measurement science that may threaten the validity and reliability of measures.
• Lack of standardization of measure concepts and specifications.
• Difficulties in implementing measures due to privacy concerns, denominator deficiency, difficult or inaccurate provider attribution, inadequate risk adjustment, or provider reluctance to participate in public reporting initiatives.
• Measure gaps, including methodological gaps and gaps in existing measures.

These methodological issues and measure gaps will need to be addressed for States and others to fully realize the potential of APCDs in increasing health care value. Some States are using their APCDs initially for public reports of State-level performance and substate (e.g., county or ZIP Code level) variations before the release of more granular analysis (e.g., by provider or payer). This approach allows early public reporting of policy-relevant data from APCDs for State decisionmakers while the barriers described above are being addressed.

Return to Contents

Conclusion

While it is clear that much remains to be learned to maximize the potential of APCDs and to reduce the difficulty and cost of using them, national momentum is building for developing measures to be used with APCDs and defining the business cases for maintaining APCDs. This report provides an overview of both the potential for APCDs to generate the information needed to improve care, as well as caveats to keep in mind while doing so.

ⁱ Visit the Institute for Healthcare Improvement Triple Aim Web page for more information.

Next Page