All-Payer Claims Databases Measurement of Care: Systematic Review and Environmental Scan of Current Practices and Evidence

Overview

The environmental scan search yielded information on existing APCDs and the potential of using APCDs to measure quality, cost, and utilization of care across populations and settings.

Literature Review

This section presents a narrative of the literature review findings. The team identified a total of 204 articles using the defined search terms. A preliminary review of the articles indicated that 98 articles met the inclusion criteria for full review. Articles were excluded if they contained no measures or no relevant information on the guiding questions. Go to Appendix B for a list of included articles, divided by Overview and Research articles.

Table 1. Characteristics of Included Articles From Literature Review

Characteristic	Number of Included Articles (N=98)	Percentage of Included Articles
Type of Article
Overview	17	17%
Research	81	83%
Type of Database*
APCD	14	14%
MPCD	39	40%
Other claims†	29	30%
Type of Measure1
Cost	75	77%
Quality	14	14%
Utilization	43	44%
Multiple	42	43%
Unit of Analysis
Population based‡	68	69%
Comparisons
Provider§	11	11%
Health plan	1	1%
Geography	4	4%

* Articles can fall into more than one category, so percentages may add to more than 100 percent. Percentages are of all articles, including overview articles, so the categories related to research (type of database, unit of analysis) may not add to 100 percent.
^† Other claims include Medicare and Medicaid.
^‡ Population based refers to measures of care across a population, such as total costs of care for all type 2 diabetes mellitus patients over a year (population aggregate), or average costs of care per member per episode of care for community-acquired pneumonia (episode costs).¹⁸ Comparisons refer to articles that present variations in costs on a population-based measure across providers, health plans, or geographic regions.
^§ Provider can refer to comparisons for any of the following provider types: hospitals, ambulatory surgery centers, clinics, or physicians.

Overview Articles

The first category of articles reviewed were “overview” articles presenting concepts related to measurement using APCDs or other large claims databases. These were commonly viewpoint or editorial articles. For example, work by Goodman, et al. (2013) and Ginsburg (2007) represent typical APCD overview articles identified from our search.

Goodman, et al., discuss the challenges of defining and measuring chronic conditions and present a conceptual model for understanding and standardizing approaches to defining, identifying, and using information about chronic conditions.¹⁹ Ginsburg describes the opportunities and challenges presented by price transparency initiatives.²⁰

Like many of the overview articles identified, these articles present concepts related to developing and applying measures to APCDs but do not present specific measures that would fit within our project scope. These articles were most often relevant to guiding questions 3-5.

Research Articles

The second category of articles reviewed were research studies that included measures used in addressing authors’ specific research questions. This category includes two types of articles: studies focused on measurement of a specific clinical scenario or episode, and articles that discuss the potential application of specific measures, such as public reporting or price transparency initiatives.

Research articles on measurement use

Articles by Gross, et al. (2013), Dalal, et al. (2011), and Mehrotra, et al. (2009) are typical examples of measure-specific studies. Gross, et al., identify the most frequent emergency department (ED) and inpatient diagnoses based on volume and cost, as well as the high utilizers of care (as defined by ED visits and cost).²¹ Dalal, et al., describe cost trends over 4 years for patients with chronic obstructive pulmonary disease, across different insurance products.²² Mehrotra, et al., describe the cost and quality of care at retail clinics for three common clinical diagnoses.²³

Given the vast number of studies that discuss measurement using administrative data, inclusion of such studies was limited to those that specifically used APCDs or APCD-like data, such as MPCDs, or that performed analyses similar to those done in APCDs (e.g., longitudinal measures and cost measures). All included measures fall into three types: utilization, cost, and quality. Some studies included multiple types of measures. Utilization and cost measures were often designed for the study to answer a specific research question; quality measures were more often established measures or based on published guidelines of care.

Research articles on price transparency and public reporting

Works by Kullgren, et al. (2013), Rosenthal, et al. (2013), and Sinaiko, et al. (2012) represent typical articles that study the use of measures for public reporting or price transparency, particularly in the context of APCDs. Kullgren, et al., describe the current status of States with health care price transparency Web sites.²⁴ Rosenthal, et al., studied the ability of consumers to obtain bundled pricing information for hip replacement surgery, data that would be available from an APCD.²⁵ Sinaiko, et al., using key informant semistructured interviews and survey methods, describe stakeholder views on why current public report cards may lack impact on consumer choice. They also present stakeholders’ suggested measurement and data collection solutions, some of which APCDs could help address.²⁶

Environmental Scan

This section presents a narrative of the environmental scan findings. The team identified 236 sources using the initial search criteria, of which 127 sources were included for full review. Documents and Web sites were excluded if no measures were included or they provided no information on our guiding questions (n=109). A total of 57 of the included sources had candidate measures.

The team did not include measures from the Leapfrog Group and CMS QualityNet, which are additional measure stewards with cost and resource measures, since they focus only on the hospital setting. The team also did not include measures from the AHRQ MONAHRQ tool, a publicly available online tool for creating and publishing a Web site for publicly reporting cost and quality, because the cost measures are also only for inpatient procedure codes. Go to Appendix C for included sources, organized by category as listed in Table 2.

Table 2. Types of Sources Searched in the Environmental Scan

	Sources Included in Scan (n=127)	Sources With Candidate Measures (n=57)
AHRQ expert materials	7	2
APCD Council materials	4	1
Reports, task forces, policy briefs	70	17
Trade, business, white papers	9	2
Books	2	0
APCD public reporting Web sites	16	16
Other measure sources	15	15
Measure inventories (e.g., NQF)	4	4

Many sources were general in nature and covered several of the guiding questions. Other sources were narrow, focusing on specific guiding questions such as measure concepts and gaps, barriers to using and reporting measures for APCDs, and methodological issues related to using APCDs for measurement. Lastly, of the sources listed in Table 2, many contained measures that were or could be used with APCD data (guiding questions 1 and 2). These sources are described in greater detail below in the section "Key Categories and Domains for Measures That Leverage Unique APCD Data."

Provided below is a broad overview of the organizations providing expertise and relevant resources in the field and illustrative examples of the resources in the environmental scan.

Organizations Providing Expertise and Relevant Resources in the Field

Several national organizations have developed expertise with APCDs or measurement and have created resources and provided support for States creating APCDs. Such organizations include:

• Agency for Healthcare Research and Quality (AHRQ). AHRQ is a government organization whose primary mission is to produce and encourage use of evidence to make health care safer, higher quality, more accessible, equitable, and affordable. AHRQ provides Federal-level coordination of quality improvement efforts, as well as knowledge and support for ACPD development through some key programs:
  • The United States Health Information Knowledge Database (USHIK) was an online, publicly accessible database. USHIK contained a repository of APCD File Submission specifications and data elements from 14 Statesⁱⁱⁱ that were downloadable for comparison. It also contained links to resources such as Core Data Elements, ASC X12 Mapping, and the APCD Showcase from the APCD Council.
  • AHRQ has also provided leadership and support for Community Quality Collaboratives, including 24 Chartered Value Exchanges, committed to improving health care quality and transparency. The collaboratives include purchasers, consumer organizations, health plans, providers, State data organizations, and State Departments of Health. At its Community Quality Collaboratives Web page, AHRQ provides tools and resources on topics such as quality and efficiency measurement, public reporting, and consumer engagement.
  • Finally, AHRQ sponsors the National Quality Measures Clearinghouse (NQMC), an inventory of evidence-based measures and measure sets. The inventory is publicly accessible and searchable and includes information on measure initiatives and measures in progress, as well as expert commentaries and tutorials on measures.
     
• The APCD Council and the National Association of Health Data Organizations (NAHDO, www.nahdo.org) are nonprofit organizations that have taken the lead in supporting and documenting current State efforts and legislative work around APCDs.
  • The APCD Council facilitates information sharing for States that have developed, or are developing, an APCD; works across States to harmonize data collection and release; and provides both technical and policy support to States developing APCDs. For example, the Council convenes a technical end-user forum to facilitate information sharing and troubleshooting for States with APCDs and is working on methods to integrate Medicare data into APCDs.
  • The APCD Council is convened by NAHDO, a group of State and private health care organizations whose mission is to improve health care data collection. NAHDO has worked with States to implement inpatient, ambulatory surgery, and ED data reporting programs and helped lead the development and implementation of standards for hospital discharge data. Along with the APCD Council, the organization has extended its reach to help States expand reporting to include all-payer claims data from commercial and public payers. In addition, NAHDO is working with the Public Health Data Standards Consortium, the APCD Council, and America's Health Insurance Plans to develop core APCD reporting standards for State-based claims data systems.
     
• Catalyst for Payment Reform is a nonprofit organization representing purchasers of health care. The organization’s primary goal is to promote and accelerate payment reform toward value-oriented care. The organization publishes report cards on States’ efforts toward price transparency at the consumer level and provides support and tools for purchasers to understand payment reform.
• The Robert Wood Johnson Foundation (RWJF) is a nonprofit foundation whose goal is, in part, to identify, explore, and spread strategies that will increase the reach and efficacy of health services and decrease costs. RWJF is a primary sponsor of the Aligning Forces for Quality initiative, which seeks to improve quality of care and reduce disparities in health care by bringing together community stakeholders. Their efforts have resulted in, for example, public reports in Oregon and Washington informing consumers about quality of care in their State. In addition, RWJF has compiled resources for States looking to improve quality, utilization, and cost measurement that includes conceptual frameworks on price and cost reporting.
• Health Care Cost Institute (HCCI) is a nonprofit organization whose goal is to provide access to health care cost and utilization data to researchers and policymakers trying to understand the factors influencing health care costs. They have four main data contributors: Aetna, Humana, Kaiser Permanente, and UnitedHealthcare. They have two sets of activities: produce a twice-yearly health care cost tracker report, demonstrating the primary drivers behind increasing health care costs; and support and provide relevant data for topical research projects from independent researchers at leading universities, think tanks, and other research organizations.
• The National Quality Forum (NQF) is a nonprofit organization that endorses evidence-based, valid measures and seeks feedback to ensure measures are meaningful and accurate. NQF-endorsed measures are considered the gold standard of quality measurement, including NQF-endorsed cost and resource use measures. The organization is also active in trying to improve measurement and its implementation, and in helping patients, physicians, and policymakers understand measurement reports. For example, NQF released a draft report in 2014 summarizing their evaluation of episode groupers. NQF has also worked with the Measure Applications Partnership, a multi-stakeholder partnership that guides the U.S. Department of Health and Human Services on the selection of performance measures for Federal health programs.

National and State Resources on APCDs

General resources ranged from describing the national landscape of APCDs to State-specific issues related to building or implementing APCDs. Overviews from national organizations were often more policy focused. For example, a 2010 report from the Commonwealth Fund summarized the current status of APCDs in the United States, with examples of APCD use;²⁷ a 2012 RWJF report on health care costs discussed using APCDs for price transparency initiatives.²⁸

State-level resource articles more often presented the case for building an APCD, summarized potential uses of APCD for measurement, and identified potential barriers to establishing an APCD. For example, a white paper from the Massachusetts Hospital Association:

• Provides a brief overview of APCD implementation in Massachusetts;
• Summarizes the advantages to hospitals, health systems, and other stakeholders of creating and using an APCD to increase efficiency, reduce cost, and improve quality of care;
• Discusses barriers to APCDs; and
• Makes recommendations to policymakers of steps needed to achieve the greatest value from APCDs.²⁹

State APCD Activities

Many of the current resources available on APCDs focus on the activities of State APCDs. For example, the APCD Council maintains an interactive map that enables users to find information on APCD activity in each State. APCD reports are the most common form of reporting of measures, leveraging the strength of APCDs, that is, the ability to study patients across time, settings, and insurance carriers.

New Hampshire, Vermont, and Massachusetts, in particular, have been active in terms of using their APCDs for measurement. For example, the 2007-2011 Vermont Health Care Cost and Utilization report describes trends in health care spending per Vermont resident, including medical and prescription spending, benchmarked against national averages.³⁰

Similar statewide reports have been generated for quality measures for specific clinical scenarios (e.g., vaginal delivery rates versus Caesarean section rates in New Hampshire³¹), as well as patient-focused cost measures such as an analysis of out-of-pocket costs across insurance products in New Hampshire.³² State-based reports, particularly from States with well-established APCDs, provide a wealth of information on the feasibility of using APCDs for measurement, including the inadequacies of the data and barriers to use.

Price Transparency Efforts

Finally, the environmental scan yielded information about the application of APCD measures—State-level reports using APCD data or public-facing reports to support price transparency. Several States have price transparency initiatives that are presented to the public via a Web site. Some States have been able to analyze their experiences with price transparency. For example, New Hampshire has had enough experience with their price transparency program to publish a report detailing their lessons learned.³³

Higher level summaries about price transparency are also available, both from governmental sources such as the Government Accountability Office,³⁴ and from groups such as the Pacific Business Group on Health³⁵ and the American Hospital Association.³⁶ Such price transparency initiatives depend heavily on APCD measures and thus are a potential source of measures for the measure inventory.

Return to Contents

Key Categories and Domains for Measures That Leverage Unique APCD Data

Guiding questions 1 and 2 focus on measures that leverage the unique APCD data. The details of individual measures that were found in the literature review and environmental scan were included in the measure inventory. This section details several key measure categories from the literature review and environmental scan that guided the creation and organization of the measure inventory (Table 3).

This table formed the basis of initial work on the inventory. The inventory includes additional information on the categories of measures (examples, potential strengths and weaknesses), and all measures in the inventory are organized to reflect these categories.

These categories clarify potential uses of the measures and the advantages of some types of measures for the different uses. The categories also assisted in prioritizing measures included in the APCD measure inventory and provided an organizing framework for the inventory. Ultimately, this framework will assist APCD sponsors in prioritizing measures for reporting and use with APCDs.

The following section delineates key categories and provides examples from the literature or the environmental scan of potential measures or sources of measures for each category.

Table 3. Measure Categories and Domains

Level of Measurement	Measure Type	Measure Domain*	Framework of Cost or Price Measure†
Individual provider	Quality	Choice	Episode of care
Medical group or Clinic	Utilization	Negotiation	Acute
Hospital	Price	Accountability	Chronic
Health plan	Costs	Population health and policy	Care components
State or region	Relative value		Population aggregate

* Measure domains are based on a conceptual framework from Painter and Chernew.^37
† These measures may or may not be risk adjusted for comorbid conditions.

In addition, based on feedback from the TEP meeting, the project team explored including a categorization related to the audience for measure reporting: consumer, provider, payer, or policymaker. It is expected that measures may have multiple audiences, and naming the potential audiences will be useful to APCD sponsors and users looking for measures that meet specific reporting goals.

Level of Measurement

Several potential levels of measurement exist for any quality measures. Historically, measures that have been used in public reports most often have been hospital-level comparisons, such as those reported in www.hospitalcompare.hhs.gov/. The level of measurement generally determines who is held accountable for improving the metric. For this project focusing on ambulatory care, measures at the level of individual physicians, medical groups, or clinics were prioritized.

In addition, the level of measurement is, in part, determined by the data elements available. For instance, an APCD sponsor seeking to report physician-level costs for treatment of type 2 diabetes needs valid physician identifiers in the database. If these are not available, a reliable clinic identifier may be used to measure costs at the clinic level.

For some conditions, measuring quality or cost at the clinic level may be reasonable (e.g., diabetes mellitus is often an interdisciplinary effort in a patient-centered medical home, involving primary care providers, nutritionists, and health coaches). For other conditions, it may not make sense (e.g., costly referral patterns may be driven by a single provider, or costs may depend on whether condition-specific clinical guidelines are followed).

State- or region-level measures could be used to inform policymakers interested in understanding the effects of policies over time. For instance, in a 2014 British Medical Journal article, Lasser, et al., compared the effects of Massachusetts Health Reform on 30-day readmissions, using APCDs from Massachusetts, New Jersey, and New York.³⁸ While 30-day readmissions are a familiar metric to those experienced in hospital-level comparisons, the risk-adjustment approach in comparing performance across States included hospital-level characteristics, such as hospital size, teaching status, nurse-to-patient ratio, and safety net status. This paper illustrates that the intended use of the APCD (e.g., making State-level vs. hospital-level comparisons) may change the level of measurement.

Measure Type

Specific measure categories are relevant when a value framework is used. It is well documented^39-41 that consumers who see cost measures alone equate higher cost providers with higher quality. Hence, in the environmental scan, the team frequently found recommendations to show quality measures next to cost measures, or to order comparative lists according to value, clearly indicating those with high quality and lower costs.^39,40,42,43

One approach to choosing measures for an inventory could be to prioritize cost measures that have potential companion quality measures. For instance, Hollingsworth, et al., in 2012 examined Medicare payments for 22 common outpatient urologic surgeries in one paper and quality measures for the same surgeries in another paper. In the payments paper, the authors reported procedure payments to Medicare.⁴⁴ Quality measures in the subsequent paper were outcome measures: 30-day mortality, unexpected admissions, and postoperative complications.⁴⁵ These types of pairings will be more useful for presenting value to consumers than papers assessing costs alone.

There is substantial literature on quality categories, starting with Donabedian's seminal work describing the structure, process, and outcome domains of quality.⁴⁶ This broad literature was not covered, but relevant to this discussion are a few points about the advantages and disadvantages of the different quality measure categories. Process measures are considered easier to capture using the kinds of administrative data that have been available historically (e.g., hospital discharge abstract databases), particularly when such data lack patient-level identifiers and therefore cannot permit longitudinal tracking of patients and their outcomes.⁴⁷

APCDs, which typically include patient identifiers, can more easily track outcome measures across time, such as readmissions, infections, and other complications.⁴⁷ Consumers usually find these outcome measures more meaningful, since they are more tangibly relevant to their health status.⁴²

Measure Domains: Choice, Negotiation, Accountability, Population Health, and Policy

In a 2012 report for RWJF, Counting Change: Measuring Health Care Prices, Costs, and Spending, Chernew and Painter describe three potential uses for measures based on APCDs. They note that "different audiences have different perspectives, needs, and capacities for understanding and using information."³⁷ The three uses are:

• Choice: managerial, consumer, or purchaser decisions.
• Negotiation (e.g., to set provider reimbursement rates).
• Accountability (e.g., global cost budgeting; public reporting to policymakers, public or private purchasers, and oversight organizations or entities such as accountable care organizations and Aligning Forces).

In addition, many States are leveraging APCD data for policy purposes, both to assess the health of their populations and to inform the design of health care reforms and assess their effects. Provided below are further details on these different uses, examples of measures in each category, and examples, either from the published literature or from the environmental scan, of how APCDs have been used to support them.

Measures That Support Choice

Choice measures might be used by consumers choosing a new primary care doctor, a specialist, or a hospital for a procedure or set of procedures. Insurers or large self-insuring employer purchasers might also use choice measures to identify preferred providers—high-value hospitals or medical groups that purchasers incentivize employees or members to use.

Using shoppable measures

Several characteristics describe measures to support choice. One useful concept is whether a condition is "shoppable." Kullgren, et al., describe good price transparency Web sites as having measures that focus on discrete episodes of care that are predictable, nonurgent, and subject to deductibles.²⁴ These shoppable conditions (elective procedures, maternity care, colonoscopy, etc.) afford the time and provide the motivation to seek and compare information.³⁹

Mehrotra, et al., provide examples of common health care choices consumers might face, including choosing a new primary care provider, selecting an obstetrician or midwife, or deciding which provider to use for an elective procedure.⁴⁸ In other situations, referrals may be primarily dictated by another physician.⁴⁸ Ginsburg, et al., delineate specific characteristics for effective price shopping situations:

1. Service is not complex;
2. Service is not urgent;
3. Diagnosis has been made;
4. Bundled payments are the norm for the service; and
5. The insurance benefit structure provides incentives to choose lower price alternatives.²⁰

Choosing primary care physicians

Primary care physician (PCP) metrics may need to be different from more discrete episodes of care such as maternity care or elective procedures. To guide choice of a PCP, information on quality and cost of managing chronic conditions might be more appropriate, using longitudinal and population-based measures. Information on common chronic conditions such as diabetes, hypertension, and obesity might be prioritized, as they would be relevant to a large number of patients and can substantially affect health outcomes.

Using the lens of consumer choice of PCP, some measures become lower priority. For instance, a published paper looks at annual cost per patient for management of genital warts.⁴⁹ Because it is unlikely that many patients would be looking for cost information on management of genital warts as a way of selecting a PCP, one might exclude this measure. An APCD sponsor could use a stakeholder process to define the conditions that would be most relevant to consumers.

Identifying types of cost and price data in consumer-facing public reports to support choice

In describing measures that support choice, the types of cost and price data available for choice are important. Kullgren, et al., reviewed 62 Web sites that enabled patients to estimate or compare prices for health care services in a State.²⁴ They report the types of data on prices that are made available to consumers on these sites: patient out-of-pocket prices; allowable charge (payment made by plan plus payments made by patients); and billed charge. Any of these categories could be reported as facility fees only or facility and professional fees.

In addition, some of the Web sites reviewed tailored cost information for patients, to provide information that was more relevant to specific individuals. The tailoring varied by Web site, showing different results based on various options, including the patient’s insurance status; specific insurance plan the patient had; and level of specific plan cost sharing (e.g., catastrophic insurance under one plan vs. more comprehensive coverage under the same plan).²⁴

Evidence shows that consumers prefer some of these data types and measures over others. For cost, consumers want to know their out-of-pocket costs for specific services (complete episodes of care) with specific providers, given their insurance benefits.³⁹ They find most useful price information that incorporates any negotiated discounts, includes all costs associated with a particular health care service, and identifies out-of-pocket costs.³⁹

As noted above, there is good evidence that reports to support choice should present both cost and quality data for comparison, in order to avoid consumers preferentially choosing higher cost providers, using cost as a proxy for quality.^39,41 An article by Yegian, et al., describes consumer preferences for types of quality measures, which can help prioritize which quality measures to pair with cost measures. Consumers prefer measures that reflect elements of patient experience or service quality, are condition or procedure specific, and are reported at the individual physician level.³⁹ Technical aspects of quality (e.g., readmissions, avoidable complication rates) should be paired with information such as patient experience to engage consumers and demonstrate relevance.³⁹

Measures That Support Negotiation

These measures are similar to measures that support choice, in that they show provider-level comparisons. Choice measures and negotiation measures differ in terms of relevant levels of comparison. The relevant level of comparison for negotiation measures would be entities with which a payer or employer might negotiate, such as a hospital, medical group, or laboratory group. Individual provider levels of comparison would therefore be unlikely to be necessary for negotiation.

Measures That Support Accountability

Accountability measures seek to hold providers accountable for the quality and efficiency of the care they deliver and are of interest to purchasers and policymakers. Many purchasers are participating in initiatives seeking to hold the health care system as a whole accountable for care spending. Policymakers want to assess whether major innovations across a geographic region are effective in improving value, including innovations such as accountable care organizations created under the Affordable Care Act.

Examples of types of measures APCDs support that might be of interest to policymakers include, at the State level:⁴⁷

• The cost of adverse health events.
• Differences in cost and utilization between the Medicaid and commercially insured population.
• Variation in provider reimbursement rates and total medical expenditures by type of service.
• Out-of-State health care migration patterns.
• Gaps in health prevention and promotion programs.
• Total cost of care for State residents.

Maine policymakers use their APCD data for this type of accountability effort. Maine has had an APCD since 2003, with data from commercial plans, Medicaid, and Medicare. Policymakers in Maine first used the data to describe drivers of high-cost care in Maine and to evaluate variations in cost and quality in the State. The State’s goal was to restructure health care delivery to create high quality and efficient systems (e.g., capacity, resource allocation, infrastructure, care coordination), particularly for accountable care organizations. The APCD data are used to assess the effects of this restructuring.

Measures That Support Population Health and Policy Efforts

Measures derived from APCDs may also be useful to support States as they craft and evaluate policies to increase access, quality, and efficiency of health care and regulate insurers. APCD data may be useful in understanding the current utilization and cost of services, as well as where care is delivered, in order to estimate the effects of proposed payment policies.⁸ For example, Maine’s analysis of APCD data found significant variation in per capita spending across health service areas for both inpatient and outpatient care. They also found that a significant portion of inpatient care was potentially avoidable.⁵⁰

New Hampshire has also used APCD data for comparison of short-term general hospital costs incurred by commercial insurance carriers in order to help the State and others understand how much hospital costs contribute to the price of health insurance premiums.⁵¹

Population health measures may address the health status outcomes of a population, whether or not health is equitably distributed in the population; determinants of health; and costs of health care for a defined population. They may also be useful in planning public health and system-level quality improvement interventions.⁵² For example, States are interested in finding gaps in disease prevention and health promotion services (e.g., what percentage of the population has had age-appropriate cancer screenings?).⁵³

Accountable care organizations may be interested in tracking utilization of services such as medication use, test results, preventive screenings, and other health services for the population within their system; and clinics may want to track disease-specific outcomes for specific physicians or the clinic population as a whole.⁵⁴ In addition, clinics and individual providers may want to compare their performance with other clinics or with State or regional benchmarks, which is a known mechanism by which quality measures have driven improvements.⁵⁵

Frameworks of Cost or Price Measures

Several frameworks are used in the literature and in existing measures to capture and convey cost or price. Described below are the most common frameworks of cost or price measures.

Common cost or price measure frameworks include episodes of care (acute or chronic episodes), population aggregate measures, and specific care components.

Acute episodes of care are discrete periods of time defined using claims that indicate a specific procedure or new diagnosis that are bracketed by clean periods (e.g., 30 or 60 days) without a claim related to the procedure or diagnosis. For instance, a hip replacement procedure could be defined as an episode of care; likewise, a bout of pneumonia. Some data analysts also speak of episodes of chronic conditions, in which the measurement is amounts paid per plan member over a specific period of time (e.g., per month or per year). Payment measures for chronic conditions such as diabetes or hypertension are typically calculated using this framework.

Population aggregate measures summarize cost over time and over a population for a certain episode or for a chronic condition.

Specific care components are discrete services provided such as the MRI done as part of an episode of care for knee replacement. These services can include laboratory evaluations, radiology or diagnostic imaging studies, and procedures. They are typically captured in an APCD using CPT codes. Episodes of care can include multiple specific care components, all related to the same overall episode.

The following sections discuss each of these frameworks in greater detail.

NQF Resources on Episodes of Care

NQF has done substantial work on episodes of care. This work resulted in two major resources:

• Measurement Framework: Evaluating Efficiency Across Patient-Focused Episodes of Care¹² and
• Evaluating Episode Groupers: A Report From the National Quality Forum.⁵⁶

The measurement framework describes the patient stages of an episode of care, from prevention in the population at risk, to evaluation and management, to followup care (including monitoring for complications or recurrences).¹² A generic episode of care involves three phases:

• Phase 1: Population at risk.
• Phase 2: Evaluation and initial management. The clinical episode beings as Phase 1 ends and Phase 2 begins.
• Phase 3: Followup care.

At appropriate times throughout the episode, the following activities take place:

• Determination of key patient attributes for risk adjustment.
• Assessment of informed patient preferences and how well care processes align with these preferences.
• Assessment of symptom, functional, and emotional status.

At the end of the episode, measures include risk-adjusted health outcomes (i.e., mortality and functional status) and risk-adjusted total cost of care.

When applied to acute care, episodes capture transitions between providers and settings. When applied to chronic conditions with preference-sensitive decisions, they capture care coordination and reflect potentially shared decisions made between providers and between providers and the patient.

The report on evaluating episode groupers describes the major issues around evaluating episode grouper software and approaches to handling those issues, which we outline below.

Limitations to episodes of care

The NQF measurement framework reports limitations to the episode approach, including:

1. Addressing appropriateness of care. Report authors describe an instance in which the Medicare Advisory Payment Commission found that Miami, Florida, appeared to be more efficient than Minneapolis, Minnesota, in terms of relative resource use per episode for coronary artery disease. However, in examining the data further, it appeared that Medicare beneficiaries were diagnosed and treated much more frequently in Miami than in Minneapolis, suggesting potential overdiagnosis, rather than more efficient care, in Florida. The suggested solution is to balance payment measurement based on episode resource use with payment measurement based on per capita resource use, or to measure the degree to which resource utilization aligns with well-informed patient preferences.¹²
2. Risk adjusting adequately. See “Risk Adjustment” in Methodology section.
3. Measuring episodes of care for patients with multiple chronic diseases. In addition to being a risk adjustment consideration, the NQF report points out that measuring episodes of care without recognizing that episodes for otherwise healthy patients may differ from episodes with multiple conditions is not a patient-centric approach, with most patients coping with more than one chronic condition.
4. Comparing organizations. Episodes of care, isolated to specific conditions, do not capture efficiencies of care within an organization that transcend disease states. If one organization cares for the same number and type of patients with fewer resources through increased efficiency (e.g., different staffing or patient flow models), the episode approach will not necessarily reflect that efficiency. Hence, the episode approach has some limitations in making comparisons between organizations.¹²

Approach to evaluating episode groupers

The NQF report Evaluating Episode Groupers points out issues that members of the Episode Grouper Evaluation Committee are confronting in evaluating episode groupers using the NQF multi-stakeholder processes. Described below are the issues that will be relevant for considering measures to include in an APCD-specific inventory.

Episode grouper software tools aggregate claims into episodes of care, using algorithms that are typically proprietary. Groupers often include hundreds of episode definitions, and different vendors use differing methods to group and attribute claims to episodes. CMS is investing in developing a grouper that will be nonproprietary and publicly available. The CMS project was part of the impetus for the NQF report on grouper evaluation.

The issues that the NQF committee points to include the following:

• The proprietary nature of the algorithms.
• The question of whether an endorsement would cover the grouper and all the conditions in the grouper or whether each condition within a grouper would be evaluated.
• The costs calculated for an episode reflect intricate decision logic embedded in the software algorithms, which create a lack of transparency and can differ substantially between vendors.

The report writers recommend that specific criteria be used in judging episode groupers: scientific acceptability, feasibility, and usability and use. Criteria that were not recommended were:

• Importance to measure and report, since the groupers have multiple uses and broad scope; and
• Evaluation of related or competing groupers, since there are substantial differences in method, design, and intended uses of the groupers, making comparisons challenging.

Episode grouper vendors who are seeking endorsement will be asked to submit data on the three recommended criteria. An overview of the recommended elements for episode grouper submission to NQF appears below, with more detail available in the original report Evaluating Episode Groupers: A Report from the National Quality Forum. The report recommends checking the criteria both at the grouper level and at the episode level. These criteria are relevant as we consider whether to include specific episode groupers in the APCD measure inventory.

• Scientific acceptability:
  • Clarity of grouper specifications.
  • Reliability testing.
  • Validity testing.
  • Severity and risk adjustment testing clearly specified with relationships that have conceptual and empiric relationships to the episode.
     
• Feasibility
  • Data completeness criteria.
  • Data availability.
  • Demonstration that the data collection strategy can be implemented.
     
• Usability and Use
  • Intended and planned uses are clearly described.
  • The benefits of using the grouper outweigh unintended consequences.
  • The grouper will be adequately maintained.

How Population Aggregate Measures Link to Episodes of Care

Population aggregate measures can be based on acute or chronic episodes of care, by calculating the aggregate costs for all episodes of care across a population. For instance, Maine’s assessment of the effects of accountable care organization innovations, described above in “Measures That Support Accountability,” might include population aggregate measures. These measures could be used to assess changes in costs over time for specific conditions or procedures.

In using population aggregate measures to compare State policies or spending, risk adjustment strategies may need to include adjusting for hospital-level characteristics in addition to patient-level characteristics. This approach is needed to account for differences in hospital payer mix or teaching status, which might affect resource utilization at the hospital level in ways State-level innovations do not.

Episodes of Care Versus Discrete Care Components

Episodes of care are potentially more consumer-friendly descriptions of care compared with discrete care components. A cost measure based on the episode of care for a torn knee ligament would combine the costs for each discrete component, such as the physician visit, the MRI of the knee, a procedure to repair the tear, pain medications, and physical therapy sessions afterward.

The alternative measurement approach used on some public reports defines each discrete component of care using CPT codes, which requires consumers to know the CPT code in order to get the correct information. In addition to the difficulty of obtaining CPT information, the CPT approach presents a cognitive challenge for the consumer. Painter, et al., describe this challenge using the analogy of an airline ticket. An airline ticket price allows consumers a quick and intuitive way to compare airlines, rather than needing to compare prices based on individual airline costs for the jet fuel, food, cabin cleaning, pilot salaries, etc. Costs based on episodes of care perform a similar simplifying function to airline ticket pricing.³⁷

In summary, listed in the box below are some major potential categories considered in creating the measure inventory.

Categories for Consideration in Developing a Measure Inventory for Use With APCD Data Purpose of publicly reporting the measure (choice, negotiation, accountability) Condition of focus (common, costly, high morbidity or mortality?) Level of choice (whether the condition is "shoppable") Availability of a quality measure to pair with a cost measure Framework for the cost measure (episodes of care vs. care components) Type of cost data (reimbursement rates, out of pocket payment, etc.) Level of measurement (e.g., regional level, clinic or medical group, individual clinician)

Return to Contents

Sources of Measures for Use With APCDs

Table 4 gives a high-level overview of the key sources of measures identified through the environmental scan and literature review. The first phase of measure inventory development entailed prioritizing measure sources, types, and focus; then details of individual measures were compiled for further review.

Table 4. Number of Measures or Public Reports for Potential Use With APCDs

Source	Number
Literature review	65 papers*
NQF Administrative Claims Measures
Ambulatory quality	143 measures
Resource	9 measures
National Quality Measures Clearinghouse
Episode measures	141 measures
Cost measures for physicians	74 measures
Public Reports
APCD public reports	7 public reporting Web sites
Other public reports with cost or resource measures†	7 public reporting Web sites or reports
Other Measure Stewards or Resources
NCQA relative resource use measures	5 measures
Quality Alliance Steering Committee	22 measures
APCD Showcase	41 reports
Bridges to Excellence	4 NQF-endorsed measures
HealthPartners	2 NQF-endorsed measures

Key: NQF: National Quality Forum; APCD: all-payer claims database; NCQA: National Committee on Quality Assurance.
* These papers provide measures or potential measures by describing one measure specific to the study question, using claims data, or describing the use of a group of measures that are already in use and are described elsewhere (e.g., NQF-endorsed measures, Centers for Medicare & Medicaid Services measures).
^† From a list compiled in Evidence-based Practice Center Technical Brief Protocol. Public Reporting of Cost Measures in Health.²

To provide guidance in selecting measures and sources to include in the indepth review during measure inventory development, provided here is a brief description of the types of measures from each source, with examples of characteristic measures from each category of sources.

Table 5 summarizes the cost or efficiency measures on public reporting Web sites using APCD or other administrative claims data.

Literature Review

As noted above, the literature review identified cost, utilization, and quality measures.

In creating the measure inventory and deciding whether to include a measure from the literature, a few other points were of interest, such as:

• Whether the measures were developed for the study or were “off the shelf,” previously vetted measures in use before; and
• What conditions were studied.

Measures that are vetted will be prioritized over measures developed for a specific study. Established measures will likely be more feasible to implement with APCD data in the near term, whereas study-specific measures may need some modification or further scientific testing for validity and reliability before being ready for use in public reporting.

The choice of condition, or of cross-cutting measures, for study will provide information regarding the portion of the population for whom the measure is relevant and whether the condition is common, costly, or associated with high morbidity or mortality. Common and burdensome conditions are likely to be salient for more consumers (see “Measures That Support Choice” above), as well as for policymakers and payers.

Lastly, some studies from the literature focused on specific choices of medication, such as a study comparing per member per month costs of two different chemotherapy regimens for a specific type of breast cancer.⁵⁷ Potential measures that could be derived from these types of papers would therefore focus narrowly on the cost of a specific chemotherapy regimen. These types of measures may not be useful given the very narrow population for whom they would be relevant and given that the episode-based approach for types of cancer is considered the more intuitive approach for the consumer.³⁷

NQF Administrative Claims Measures of Ambulatory Care

From the NQF measure Web site, a set of measures that use administrative claims data and focus on ambulatory care were identified. These measures are either resource use (cost/utilization) measures or quality measures.

Of the 9 resource use measures, a limited number were specific for costs (e.g., NQF #1609, Episode Treatment Group (ETG)-Based HIP/KNEE REPLACEMENT cost of care measure). The others measured relative resource use rather than dollar amounts.

Of the 143 quality measures, some require electronic medical record data in addition to administrative claims data (e.g., “Osteoporosis: Communication with the Physician Managing On-going Care Post Fracture of Hip, Spine or Distal Radius for Men and Women Aged 50 Years and Older,” which requires medical record documentation access to assess for numerator status). Because NQF has reviewed and endorsed these measures, their scientific strength (validity and reliability) has already been evaluated.

National Quality Measures Clearinghouse

NQMC is a database and Web site for information on specific evidence-based health care quality measures and measure sets. As noted above in the section on AHRQ resources, NQMC is sponsored by AHRQ to promote widespread access to quality measures by the health care community and other interested individuals.⁵⁸

The project team searched the NQMC database using the following keywords and data source and measurement setting filters, resulting in 215 measures to be reviewed in greater depth for the measure inventory (see box below). Based on a preliminary review of these measure titles, the team anticipated that some would be excluded because they required more than just administrative data (e.g., survey data; the filters are inclusive but not exclusive).

National Quality Measures Clearinghouse Searches Search 1: Keyword: episode Data Source: administrative clinical data, administrative management data Measurement Setting: ambulatory/office-based care, emergency medical services, emergency department, hospital outpatient, patient-centered medical homes, ambulatory procedure/imaging center Results: 141 Search 2: Keyword: cost Measure Initiatives: Physician Quality Reporting System Results: 74

APCD Public Reports

Seven States have public reporting Web sites that use APCD data to report on price and quality. Table 5 presents the types of measures available on these Web sites, summarizing clinical conditions of focus, level of measurement, framework of the cost or measurement data, and types of cost or quality measures shown, if any.

Other Public Reports With Cost or Resource Measures

Other public reports with cost or resource measures included reports from the Chartered Value Exchange efforts supported by AHRQ and the Aligning Forces communities supported by RWJF. These public reports were listed as candidate Web sites in a Research Protocol for a Technical Brief on Public Reports of Cost for AHRQ. Those reports that only reported inpatient hospital information were not reviewed. Go to Table 5 for a summary of these public reports.

Robert Wood Johnson Foundation (RWJF) Search Tool. Another source of potential report cards was an online directory, created and maintained by RWJF, containing 208 national and State-level public reports of quality or cost for hospitals or physicians. Several issues were identified:

• Many reports only provided hospital quality information,
• Some reports were proprietary (e.g., Cigna health provider directory),
• Many only reported Consumer Assessment of Healthcare Providers and Systems (CAHPS) data or Healthcare Effectiveness Data and Information Set (HEDIS) measures, and
• Some reported on whether providers met criteria for certain recognition programs (e.g., Bridges to Excellence shows icons for recognition programs but no additional information), which cannot be assessed with administrative data only.

Four of these Web sites that were not excluded for one of the above reasons were reviewed and are listed in Table 5.

Fair Health Cost Consumer Lookup. An online public reporting Web site of costs, sponsored by Fair Health, was founded as a result of a lawsuit in New York against insurers for conflicts of interest in calculating out-of-network reimbursement rates. Go to Table 5 for details about the online report. Several public reports of quality link to this Web site in their section mentioning health care costs. For instance, Michigan’s public report links to the Fair Health tool from the “Consumer Resources” page. Minnesota’s and Ohio’s public reports also have a link to this resource.

Other Measure Stewards or Resources

National Committee for Quality Assurance (NCQA) Relative Resource Use Measures. In addition to the commonly used HEDIS measures of outpatient care,⁵⁹ the NCQA has a set of measures focusing on relative resource use.⁶⁰ These measures are health plan-level comparative measures of resource utilization that do not report actual prices but use relative value units instead. In this way, a consumer may be able to use them in choice without dollar amounts being disclosed, which some plans are reluctant to do (see the section ” Barriers to the Use of APCDs for Measurement,” below).

Relative resource use measures indicate how intensively a plan uses physician visits, hospital stays, and other resources to care for members with one of five chronic diseases: cardiovascular disease, chronic obstructive pulmonary disease (COPD), diabetes, hypertension, and asthma. They are intended for use with HEDIS quality measures, to help members, plans, employers, and benefit managers make informed choices about health care services.⁶¹

Quality Alliance Steering Committee (QASC). QASC is a multi-stakeholder group of leaders in performance measurement and quality improvement whose goal is to make consistent and useful information about the cost and quality of care widely available. QASC sponsors the High Value Health Care Project, which undertook to develop a set of 22 episode-based cost measures, with technical specifications.

The measures focus on 12 common and costly conditions:

• Acute myocardial infarction (costs for the first 30 days, and the following 31-365 days),
• Diabetes (annual costs),
• Congestive heart failure (1 year chronic management; 4 months post-hospitalization),
• Coronary artery disease (1 year chronic; 1 year post-revascularization),
• Lower back pain,
• Pneumonia (ambulatory episode; pneumonia hospitalization episode),
• Breast cancer (60 days prior to biopsy; 15 months after new diagnosis),
• Colon cancer (colonoscopy—21-day period) episode for localized cancer,
• Asthma (chronic over 1 year),
• COPD (chronic over 1 year for stable, unstable),
• Gastroesophageal reflux disease (chronic over 12 weeks; chronic over 1 year), and
• Sinusitis (acute; chronic).⁶²

QASC intends to propose these measures to NQF for endorsement.

APCD Showcase. The APCD Council has posted 41 case studies to demonstrate how States have used existing APCDs for reports of health care status. Some of the public reporting Web sites for the State APCDs are described in Table 5.

Some reports analyze patterns of price and utilization that may be amenable to reporting using other APCDs. For instance, in June 2008, New Hampshire published a report evaluating cardiovascular diseases and other circulatory disorders in adults age 19 and over. Medicaid and commercial administrative eligibility and claims data were used to study disease prevalence and associated utilization and payments. In June 2013, the Dartmouth Atlas group analyzed APCD data from Maine, New Hampshire, and Vermont to describe variations in pediatric hospitalizations, ambulatory care, common surgical procedures, diagnostic imaging, and prescription drug use.

Bridges to Excellence. The Health Care Incentives Improvement Institute (HCI3) is a nonprofit organization that has developed a payment system (Prometheus systems) and a recognition program called Bridges to Excellence for providers that deliver high-quality care for lower costs. The measures Bridges to Excellence has developed are a suite of measures of Potentially Avoidable Complications. Four of these measures are NQF endorsed. While three focus on hospitalizations, the fourth focuses on outpatient care: “Proportion of patients with a chronic condition that have a potentially avoidable complication during a calendar year.”

HealthPartners is a Minnesota-based, not-for-profit integrated health care system, founded in 1957. As part of its mission to achieve the Triple Aim as an organization, it uses a set of cost and utilization measures to be viewed alongside quality measures, in a public report of quality and cost performance of its medical groups. Table 5 describes the Web site in further detail. Two of the measures have been endorsed by NQF and are titled “Total Cost of Care Population-Based Per Member Per Month Index” and "Total Resource Use Population-Based Per Member Per Month Index."⁶³

In summary, a variety of sources of measures for use with APCDs focus on cost or quality. The literature provides studies using individual measures, some with well-described technical specifications. NQF and NQMC are measure aggregators and provide access to structured technical specifications.

Public reporting Web sites either explicitly use State APCD data or use a combination of several data sources and measures, some of which can be adapted for use with APCD data. However, technical specifications are not always easily available through the online resource. Lastly, several organizations are contributing discrete groups of claims-based measures, such as NCQA, QASC, and Bridges to Excellence, who have made the technical specifications publicly available or available on request.

Table 5. Types of Cost and Efficiency Measures and Data Used on Current Public Reporting Web Sites Using APCD or Other Claims Data

State	Focus	Level of Comparison	Cost Framework	Cost Data	Other Measures	Choice, Accountability, Negotiation
APCD Public Reporting Web Sites
Colorado (Part 1 of Web site)	Maternity care Elective procedures (e.g., hip, knee replacement)	Hospital	Acute episodes of care.	Allowed costs for insured, private or Medicaid. Charges for uninsured.	Quality measures (patient experience, AHRQ IQIs and PSIs).	Choice, Negotiation
Colorado (Part 2 of Web site)	Chronic disease management (asthma, diabetes mellitus)	County or ZIP Code	Chronic episode (annual); total cost of care and total cost of care/expected total cost of care.	Dollars paid by gender and by age group. Also aggregate costs by geography. Inpatient, outpatient, facility fees, professional fees, ancillary, and prescription costs.	30-day all-cause readmissions; ED visits; diabetes prevalence; asthma prevalence; illness burden; percentage generic prescriptions; utilization and access measures.	Accountability
Massachusetts outpatient http:/ /www .mass.gov/myhealthcareoptions (no longer active)	Chronic conditions Preventive health Medication management Monitoring, acute care (strep throat testing and upper respiratory infection)	Medical group	No cost information reported for the medical group conditions.	No cost information reported for the medical group conditions.	HEDIS quality measures.	Choice
Massachusetts hospital-based measures http:/ /www .mass.gov/myhealthcareoptions (no longer active)	Maternity care Medical Radiology Surgery Patient experience Patient safety	Hospital	APR-DRG-based cost measures, using APCD data.	Relative cost, using star ratings, for procedures.	Volume of care, percentage of patients with major severity of illness.	Choice
Wisconsin	Family medicine Internal medicine Pediatrics	Clinic	No details given on cost measure framework.	“Makes good use of healthcare dollars” composite. Based on APCD claims data, no further details on the site.	“Provides Recommended Care” composite. Multiple conditions, but reports only three performance levels on composite.	Choice
Maine	ED visits Laboratory tests Office visits Outpatient procedures Radiology	Hospitals and ambulatory centers	CPT codes (note: CPT codes categorize visits [ED or office visit] by complexity of decisionmaking, not by diagnosis).	Average professional cost, facility cost, total cost (all as allowed amounts). All from APCD data.	Patient complexity level, distance to facility, number performed. No quality measures.	Choice, Accountability
New Hampshire	ED visits Office visits Outpatient procedures Radiology	Hospital, clinic, ambulatory surgical center	CPT codes (note: CPT codes categorize visits [ED or office visit] by complexity of decisionmaking, not by diagnosis).	Uses information on insurance carrier and plan, and incorporates deductible and co-insurance. Creates: estimate of what consumer will pay,estimate of what insurer will pay.	No quality measures. Precision of the cost estimate, typical patient complexity for each CPT code.	Choice, Accountability
Virginia	Preventive health ED visits Radiology Maternity care Surgery	State averages only, by provider type (e.g., ambulatory center vs. hospital outpatient)	Unclear. Potentially CPT codes. Average allowed amount.	Average allowed amount across all plans, divided into facility fee, professional fee—radiologist, surgeon, physician).	No quality measures.	Choice, Accountability
Washington	Many process measures; unclear which use APCD data and which do not	Medical group or clinic	None.	None.	Many process measures (screening, vaccinations, etc.).	Choice, based on quality only
Other Public Reporting Web Sites
Kansas HealthyKC	Asthma hospital costs and utilization	All region	Unclear.	Stratified by insurer and race and insurer and age.	HEDIS measures and CAHPS measures of patient experience.
Maine	“Working to control costs” (Yes means that the practice is working with “Get Better Maine” to achieve high-quality care at low cost)	Clinic	None.	None.	HEDIS measures and CAHPS measures of patient experience. Accepting new patients.	Choice
California PacifiCare	Cross-cutting measures (e.g., readmission rates) Condition-specific avoidable admission rates	Plan level	None, efficiency measures only.	None.	Efficiency measures: 30-day overall readmission rate, avoidable admissions, average inpatient length of stay, ED visit volume, outpatient surgery visit volume. Volume visits are per 1,000 members per year.	Choice, Accountability, Negotiation
Illinois	Medication management	Clinician or practice	Prescription data.	Percentage generic prescriptions.	Flu vaccine given, colonoscopy measures, HEDIS.	Choice
Minnesota and Wisconsin HealthPartners®	Diabetes medication adherence Total costs of care (primary and specialty care) Medication management	Practice	Prescription and cost data, unclear sources.	Unclear, although some are HEDIS measures.	Multiple, related to quality of care for other diseases (behavioral health, diabetes) and domains (safety, medication monitoring). Use additional datasets aside from claims.	Choice
Wisconsin	Many process measures; unclear which use APCD data and which do not	Medical group or clinic	None.	None.	Many process measures (screening, vaccinations, etc.).	Choice, based on quality only
Fair Health	Many conditions	Geographic area defined by the first three digits of the ZIP Code	CPT code-based charges and out-of-pocket cost.	For uninsured, shows charges only. For “insured” (not more specific), shows charges, reimbursement, and out-of-pocket costs. Also shows Medicare costs.	No quality measures.	Other: Financial planning

Key: AHRQ IQIs and PSIs: Agency for Healthcare Research and Quality Inpatient Quality Indicators and Patient Safety Indicators; ED: emergency department; URI: upper respiratory infection; APR-DRG: all patient refined diagnosis related groups; CPT codes: Current Procedural Terminology codes; HEDIS: Healthcare Effectiveness Data and Information Set; CAHPS: Consumer Assessment of Healthcare Providers and Systems.

Return to Contents

Barriers to Use of APCDs for Measurement: Data Completeness, Quality, Standardization, and Access

Although APCDs are meant to contain comprehensive claims data across settings and time, efforts to use APCDs for measurement have been hampered, in many cases, by issues with data completeness, quality, standardization, and access. Identifying and resolving those barriers is critical to the ability to use APCDs to improve health care value. The most salient issues are discussed below and summarized in Table 6.

Table 6. Key barriers and potential solutions relating to data availability, quality, and access

Barrier	Potential Solution
Missing data elements	Develop separate submission mechanisms for certain information (e.g. plan benefit design). Add fields that indicate non-claims-based information (e.g., medical home or capitated arrangements). Develop public report card for completeness of data submissions by payer.
Low data quality	Develop public report card for data quality. Improve data infrastructure.
Lack of data standardization	Establish industry standards (e.g., ANSI ASC X12, National Council for Prescription Drug Programs). Establish standard reporting frameworks.
Difficulty with data linkage and aggregation	Establish master patient index and master provider index. Consistently use National Provider Identification (NPI) numbers.
Lack of data access and availability	Improve data reporting.
Policy barriers and resource limitations	Diversify funding sources. Work toward harmonization of stakeholder interests.

Barrier 1: Missing Data Elements

APCDs are currently primarily built on administrative claims data submitted by health plans (preferably with Medicare claims as well). However, several factors affect the completeness of the data that are submitted. First, even in States where data submission to the APCD is mandatory, there are some exceptions to who needs to submit data and what data are submitted.⁶⁴ For example, APCDs usually do not have data on uninsured patients, since all payments are out of pocket and no claim is generated. There are also restrictions regarding data from potentially sensitive claims, such as those related to behavioral and mental health, HIV, and worker’s compensation.

Furthermore, certain health plans are often not required to submit data to APCDs. These include Tricare, Department of Veterans Affairs, Federal Employees Health Benefit Plan, Indian Health Service, pension plans (ERISA), and small private insurers who do not meet the minimum data threshold (e.g., 10,000 covered lives). Data from large, self-insured employers may also not be accessible, unless through a third party administrator of the health plan. In addition, for health maintenance organizations and risk contracts in which no claim is generated, encounter data are often very different in nature and completeness than claims data.

Because there are no national and often no State standards for data submission to APCDs, data element availability varies across health plans and across States. In addition, many data elements that may be collected by health plans are not reported to the APCD (Table 7).¹ For example, no information on plan benefit design or non-claims-based payments (e.g., pay for-performance payments, medical home payments, or settlements) is captured in an APCD.

Provider information is also often limited, and building comprehensive provider directories is not straightforward. Provider affiliation with a medical group may not be captured and attribution to nonphysician providers is difficult, as services provided by nurse practitioners, for example, may only be billed under a physician’s identification (go to Barrier 4: Difficulties With Data Linkage and Aggregation for further discussion about provider attribution).

Furthermore, data are typically limited to claims only. Any other data sources, such as public health data, electronic health record data, and aspects of hospital care that are part of a bundled payment (e.g., medications) that are not captured within submitted claims are not present in the APCD. Thus, some aspects of clinical care such as symptom severity or individual treatments administered during hospitalizations may not be measurable using current APCD data. Since one of the main benefits of APCDs is to perform population-based analyses, the missing data elements also affect accuracy of measurement, particularly in terms of estimating utilization or cost across populations and settings.

Some potential solutions addressing the above-mentioned barriers have been proposed. States could develop separate submission mechanisms or add labels in their database for certain types of information, such as health plan benefit design information, designations of medical homes, and capitated payments.⁶⁵  Finally, States or external organizations could develop public report cards for the completeness of data submission by insurers.⁶⁶

Table 7. Common Included and Excluded Data Elements in State APCDs

Information Typically Collected in an APC	Data Elements Typically Not Included in an APCD
Encrypted SSN or member ID number	Services provided to uninsured
Type of product (HMO, POS, indemnity etc.)	Denied Claims
Type of contract (single person, family, etc.)	Workers compensation claims
Patient Demographics	Premium information
Diagnosis, procedure, and national drug codes	Capitation fees
Information on service provider	Administrative fees
Prescribing physician	Back end settlement amounts
Health plan payments	Referrals
Member payment responsibility	Test results
Type and date of bill paid	Provider affiliation with group practice
Facility type	Provider networks
Revenue codes	Health plan benefit design information
Service dates	Master provider ID system

Key: SSN: social security number; ID: identification; HMO: health maintenance organization; POS: point of service.
Source: Porter J, Love D, Peters A, et al. The basics of all-payer claims databases: a primer for states. Princeton, NJ: Robert Wood Johnson Foundation; January 2014. Accessed February 1, 2017.

Barrier 2: Low Data Quality

In addition to missing data elements, the quality of data submitted to APCDs can vary, threatening the ability to obtain accurate measurement using APCD data. Two types of errors, in particular, affect the accuracy of the data submitted to APCDs.

First, APCDs rely on individual health plans to collect and report relevant data and check data quality. Any misreporting or inconsistent data recordkeeping related to poor clerical or data processing protocols results in inaccurate data being reported to the APCD.⁶⁷ For example, payers may bundle and unbundle claims repeatedly during claims adjudication and editing, introducing errors in the process.⁶⁸ In addition, payers may not keep track of all the data collected and may not have adequate processes for data quality checks before submitting their data, and States may have limited resources to check submitted data for accuracy. Finally, even when States do check data quality, it can take several rounds of corrections before data are accurate, which can affect the timeliness with which data are available for analysis.⁶⁷

Second, errors can occur in reporting data by or about providers. For example, some providers may systematically upcode certain aspects of care, hindering accuracy of analysis for those conditions.⁶⁸ Still others may only collect certain kinds of data only once per year, rather than continuously. For example, Massachusetts originally collected data each month on an individual’s primary care physician for the purposes of attribution. However, the monthly report proved to be burdensome for plans, so Massachusetts changed its reporting requirement for primary care physician attribution to once a year, at the end of the year.⁶⁵ While this time interval is easier for plans to submit and the State to collect, it may not reflect accurate attribution of services to a primary care provider, if that provider has changed during the year for a specific patient.

Improving data infrastructure to address the above issues is a challenge, as States must coordinate submissions among multiple plans. However, some States have begun to put processes in place to try to improve data quality. For example, Massachusetts performs the following analyses on their APCD to determine relative accuracy and completeness.⁶⁹

For data completeness:

• How many covered lives are included in the APCD?
• What percentage of Race, Ethnicity, and Language fields are being reported by health care payers?
• Does the APCD capture services provided by nurse practitioners?

For data accuracy:

• What is the distribution of male and female Massachusetts residents in the APCD?
• What limitations does a less than 5 percent reporting of Race, Ethnicity, and Language fields have on tools for indirect estimation?
• How are claims-based cost analyses affected by the reported distribution of services performed by nurse practitioners?

In addition, some organizations have proposed that public report cards be issued for data quality in order to hold plans accountable for submitting accurate data.⁶⁶ Such report cards, however, may be costly and difficult to implement.

Barrier 3: Lack of Data Standardization

In addition to data quality issues, the lack of data standardization represents a significant barrier to the use of APCDs for measurement. Currently, there are no national standards for collecting and reporting insurance claims and eligibility files for medical and prescription drug coverage. Therefore, data are inconsistent across payers; payers do not necessarily collect the same data fields, and data are often collected by different methods and with different definitions, making it difficult to aggregate. For example, collection of health disparities-related fields such as race, ethnicity, and language are not standardized.⁶⁶ Most collection is done by observation, rather than self-report, by administrators and health care providers not trained to collect such data. This method hinders the ability to conduct analyses related to health equity using an APCD.

The lack of standardized reporting fields makes comparisons across States difficult and, for health plans that operate in multiple States, creates an additional burden of submitting data in different formats, which may increase the cost of data submission.⁸ Finally, no national standards exist in terms of codifying certain fields, such as plan benefit design, into a common coding system.⁶⁵ Thus, even if the APCD were to collect such data, analysts and researchers would face significant barriers in using the data.

Recognizing that lack of data standardization is a significant barrier to use of APCDs for their intended purpose of measurement, the APCD Council has engaged two data standards management organizations to assist in establishing national standards for data.⁷⁰ The American National Standards Institute (ANSI) ASC X12 has been chosen to assist with developing standards for submission of medical claims. The National Council for Prescription Drug Programs has been tasked with developing standards for submission of pharmacy claims.

In addition, establishing standard reporting frameworks could assist with increasing the usability of APCD data. Standardized data fields and standard definitions would help ensure that a particular data label has the same meaning across plans and across States. Such efforts are critical to meaningful use of the APCD for measurement.

Barrier 4: Difficulty With Data Linkage and Aggregation

To maximize the potential of APCDs for measurement, data must be linked in a way to allow analyses of episodes of care. Currently, due to concerns about security of the protected health information of individual patients, many health plans are reluctant to report anything other than de-identified data.⁷¹ That can lead to problems linking patients, as the linking variable is often a unique identifier used only for that year and that database.

Linking to other databases is usually not possible at the patient level. Linking problems are particularly likely to occur if the episode spans the end of a year or data reporting period. Even if the patient stays in the same plan, the next data submission may have a separate set of patient identifiers. In addition, patient enrollment in plans is not stable year over year, so episodes of care that span a change in plans may not be captured.

These limitations reduce the ability to draw meaningful conclusions about quality of care, particularly for chronic conditions where patients are seen and managed by several physicians over time. Finally, as data submission requirements change over time, longitudinal evaluation of trends may be compromised by the complexity of cross-walking data elements between versions of data.⁶⁸ These changes would particularly affect the ability to track year-over-year trends in utilization and cost.

An additional barrier to tracking patients across providers is inconsistency in reporting of providers’ identification numbers, whether they are license numbers, National Provider Identification (NPI) numbers, Taxpayer Identification Numbers, or Social Security Numbers.⁶⁷ A uniform set of numbers may not be available or may be inaccurately reported for all servicing providers on claims. Providers may use different identifiers with different plans or even for different services within a plan (e.g., a cardiologist may use an individual NPI or a cardiology group NPI for an outpatient cardiac catheterization but a hospital license number for an inpatient cardiac catheterization).

Another issue is that mid-level practitioners such as nurse practitioners and physician assistants may not have separate NPI numbers. They may file claims under a supervising physician’s number.⁷² Thus, it would be impossible to separate which services were provided by a physician and which were provided by a mid-level practitioner. In addition, NPIs may refer both to an individual practitioner and a billing practice, and again, there may be confusion over which individual provider provided the services billed. Consistent use of NPIs or creation of a Master Provider Index for the situations defined above, for example, would allow easier and more accurate data aggregation.

The above challenges may also be addressed in part by creating a Master Provider Index at the highest level of data aggregation, most commonly the State level. Massachusetts, for example, has created a Master Patient Index⁷³; however, since this index was only instituted in the 2014 reporting year, the effects of the Master Patient Index on the usability of APCD data are not yet clear.

In addition, a Master Provider Index or Master Patient Index at the State level does not address the issue of patients crossing State lines for care. Crossing State lines for care may happen more often in small States with limited provider workforces and border areas where a single metropolitan area spans a State line. Also, patients may seek tertiary or quaternary care outside their typical service provider area. Creation of a universal patient identifier at a national level could result in greater ability to track patients over time and across geography; however, the cost to implement a universal patient identifier is estimated to be $1.5 billion to $11 billion.⁷⁴

Finally, for price transparency initiatives, claims need to be bundled in a consistent way so that “event” prices are comparable from one hospital to another and one health plan to another.³⁵ For example, for estimates of total cost for total hip arthroplasty, some hospitals were able to provide a “bundled price” (including hospital and physician fees) and others quoted prices that only included hospital fees or physician fees.²⁵ Thus, without consistent definitions of an “event,” it would be difficult to give accurate pricing on a public reporting Web site.

Barrier 5: Lack of Data Access and Availability

Currently, 16 States mandate reporting of data to APCDs for most health plans. In the few States where reporting is voluntary, however, APCDs may face barriers to getting access to private data. Insurance companies and hospitals often are not comfortable releasing data, primarily because of privacy guidelines and concerns. For instance, Connecticut does not mandate reporting of hospital data, so the Connecticut APCD does not contain complete hospital data, making it difficult to accurately measure quality, utilization, and cost at the State level.⁶⁶ Institution of reporting mandates, however, is a political decision usually made by the State legislature and requires the agreement and cooperation of multiple stakeholders (e.g., consumers, health plans, State legislators).

Timely availability of data is also a barrier to maximizing the potential of APCDs for measuring quality, cost, and utilization. However, claims data are not available in real time. For example, there is a significant time lag to obtain Medicare data from the Centers for Medicare & Medicaid Services (1-2 years). Thus, many population-based analyses that require both commercial health plan and Medicare data may not be able to be done in a manner timely enough to affect current care.⁷⁵

A review of Maine’s APCD processes revealed that only 50 percent of claims are adjudicated within 1 month of the service. The number rises to 85 percent after 3 months, but since the current release schedule of data is 90 days after close of the quarter, about 15 percent of claims from that quarter may not be available at the release of data.⁶⁸ For rapid cycle improvement, producing performance assessments quickly is important, so APCD sponsors and users will have to balance timeliness versus completeness.

Barrier 6: Policy Barriers and Resource Limitations

As discussed above, many of the barriers to maximizing the potential of APCDs are related to the need to improve the data infrastructure. For example, data collection and submission can be expensive and in States without mandatory data reporting, health plans may not want to or may not be able to submit data due to cost concerns. States, similarly, may lack the funding to do extensive data cleaning and checking, particularly as required with Medicare and Medicaid data.

The cost of establishing and maintaining an APCD and publishing and analyzing database information can be significant, between $500,000 and $1.8 million per year.⁶⁴ Funding is usually a mix of Federal, State, and nongovernmental resources, with typical funding sources including general appropriations from the State legislature, fee assessments, Innovation Models grants from the Center for Medicare & Medicaid Innovation, foundation funding, and data sales and subscriptions.^27,67 Diversification of funding through several mechanisms may be more sustainable in the long run; however, particularly with State funding, a good business case is required for funding to continue to be appropriated.

Political obstacles can also prevent stakeholder groups from reaching consensus on improving and expanding APCD infrastructure. For example, providers may object to payers reporting data about their practices and may doubt that the data will accurately reflect prices and quality.⁶⁴ Also, providers currently have few incentives to share their data⁷¹ if they are not in a mandatory reporting State, because very few clinicians or administrators are trained in how to use data and information to identify and address gaps in care.⁶⁸

Large, multistate insurers may become concerned about the administrative costs of complying with various State database requirements, particularly in States with strict medical-loss ratio standards.⁶⁴ Finally, consumers may be concerned about the privacy and security of their information, despite numerous safeguards in place.²⁷ Therefore, it may be difficult to reach consensus on the best ways to expand APCD data collection and use.

Return to Contents

Methodological Issues Pertaining to Use of APCD Data for Measurement

One of the main advantages of APCDs are that they are sources of comparable, population-based data that can support measurement at multiple levels of care (population, health plan, hospital, provider, and patient), across settings, and longitudinally across time. Different types of measures can be supported through APCD data, including utilization, cost, quality, efficiency, and population health measures.

All existing State APCDs currently include utilization measures, and most also address cost of care. Quality, efficiency, and population health measures are somewhat less prevalent in the current landscape, likely in part due to stakeholder interests. Several States also publicly report measures using APCD data. As more States develop APCDs and public reporting Web sites based on APCD data, it is critical that sites contain valid measures relevant to stakeholders and feasible to implement on a large scale.

Discussed below are potential methodological issues in using APCDs for measurement.

Key Methodological Issues for Measurement Using APCDs Inadequate measurement science that may threaten the validity and reliability of measures Measure specifications: Lack of standardization of measure concepts and specifications Measure implementation: privacy concerns, denominator deficiency, difficult or inaccurate provider attribution, inadequate risk adjustment, and provider reluctance to participate in public reporting initiatives Measure gaps: methodological gaps and gaps in existing measures

Issue 1: Inadequate Measurement Science

Ensuring measure validity and reliability is a critical step to ensuring successful measurement and price transparency programs using APCD data. States with active APCD measurement programs have guidelines governing the choice of measures implemented using APCD data.

Examples of guidelines include:

• Using nationally accepted standard measure sets when possible,
• Requiring that a measure has empirical evidence that it is a valid representation of the dimension of care it purports to represent, and
• Requiring that a measure provides stable and reliable information and that the measured entity is associated with a significant amount of the variance in the measure.¹⁴

However, in some areas, nationally accepted standardized measure sets are not yet widely available, and there are challenges in developing validated measures.

Developing cost measures is particularly difficult, because the definition of cost varies. For example, some cost measures may use charges and others use payments or measures of relative resource use, such as relative value units (RVUs).¹⁴ In addition, cost metrics vary according to the perspective of the user.⁷⁴ Cost to a health plan is the cost of production plus a provider’s overhead margin. On the other hand, cost to purchasers and consumers includes health plan cost plus the health plan’s margin. Each different frame yields a different answer on cost of care. In addition, fragmentation in the health care system across all levels can hinder the implementation of meaningful cost metrics.

Standardizing the data collection for cost metrics at a high level (State or national) can help address such issues. Other innovative approaches come from the research literature. Levit, et al. describe a method of calculating price-to-charge ratios rather than the more common cost-to-charge ratio, validating the consistency of this approach with data from Medicare and Marketscan commercial insurance data.⁷⁶ Speir, et al., describe an approach to calculating the additive cost of complications after coronary artery bypass surgery.⁷⁷

Reliability of measures is also an important part of evaluating whether a measure is scientifically acceptable. Since APCDs contain claims data, some inherent unreliability of the data occurs due to factors mentioned above in the section on barriers to using APCDs for measurement (missing data, lack of data standardization, etc.). Another challenge to reliability is ensuring that sample sizes are sufficient for accurate reporting at the level chosen. Sample size is particularly difficult for provider-level measures, which are discussed in more detail below.

Finally, measures need to reflect something meaningful to providers or patients and respond to interventions intended to improve care, to encourage providers to participate and engage in quality improvement.¹⁴ Chosen measures should also show evidence of a significant amount of variation at the measured level.

Some quality measures in use have little or no variation across providers and therefore are not very helpful in discerning differences. For example, the Joint Commission's core hospital measure for acute myocardial infarction includes prescribing aspirin at discharge. This measure has a compliance rate of 99 to 100 percent, so it cannot be used to distinguish high-quality from low-quality hospitals for acute myocardial infarction care.

Issue 2: Measure Specification

Even if valid and reliable measures are available for use with APCDs, one of the biggest barriers is variation in measure specifications, even for the same measure concepts. For example, the Medicare Shared Savings Program and the Medicare Advantage Star Ratings both require measurement of prevention, patient experience, and safety. However, the Medicare Shared Savings Program rates providers based on 33 measures, while the Medicare Advantage Star Ratings use 36 different measures to rate provider quality in these areas.⁷⁴

Harmonizing measures is challenging due to a lack of consensus about the best measures to use. Establishing core metrics that are useful for multiple purposes such as clinical care, population health management, reporting, and payment programs may also be helpful.

Additional barriers exist in terms of developing measure specifications for cost measures. As noted above in the discussion of limitations to episodes of care in the section "NQF Resources on Episodes of Care," calculating the cost for an episode of care is complex. It is difficult to specifically determine whether services are related to the episode (even when there is a diagnosis code suggesting it is). For patients with multiple chronic diseases, defining which “episode of care” to use may be difficult.

In addition, most cost measures are site, service, episode, or condition centric, which may not consider the total cost of care for a patient over a period of time. Development of global payment measures, such as bundled payments and patient-centric measures, may help standardize cost measurement.³⁷

Issue 3: Measure Implementation Issues

Implementation of measures presents a variety of challenges, including concerns about privacy, denominator deficiency, provider attribution, risk adjustment, and actionability. Providers, in particular, may not share data if they do not believe that the measures being reported accurately reflect their practice.

Data privacy and security present challenges to successful implementation of a measurement program, because health plans may not want to share proprietary information, and providers and patients may worry about the public release of individual-level data. All States with APCDs have data security protocols that may include not collecting direct identifiers and adopting encryption methodologies. Some, such as Minnesota, do not release their data to external organizations.^1,8

Data security protocols, while effective in protecting privacy, may limit the usability of data for measurement, particularly for researchers or for making comparisons across States. One approach to increasing data usability while protecting privacy may be to use distributed data models, in which individual-level data stay with the data owner but aggregated information and measures can be submitted to the State.^5,74

As discussed above, one barrier to accurate measurement, particularly at the practice level, is ensuring that each provider has a sufficient denominator. Individual physicians see relatively small numbers of patients with any particular condition. These numbers can be reduced because the claims data used to generate quality metrics are collected and maintained separately by different health plans.⁷²

APCDs themselves, given data submission from multiple health plans, may help overcome the sample size issue with providers. Without a universal provider directory, however, this benefit cannot be realized, because it is not possible to attribute a provider’s patients from different plans to that individual provider. To avoid the small denominator problem, one also could choose measures for common conditions and implement measures at the plan, community, or State levels.

Provider attribution for measurement has also proven to be a challenge. As discussed above in the data aggregation section, provider identifiers are often missing or inaccurate or plans have very loose rules about which identifiers can be used. Attribution methods that can be applied to both cost and quality measures also need to be developed, to ensure that a physician’s performance in those two areas is assessed based on the same panel of patients.⁷²

One of the main challenges to using APCD data for measurement is appropriate risk adjustment so that comparisons can be made across populations and providers. Measures need to be risk adjusted to avoid disincentives to treat more complex, costly patients, particularly patients with multiple chronic conditions. Risk adjustment is particularly important for cost and utilization measures, as it can explain about half of all practice-level variation.⁷⁸

Various methods can be used for risk adjustment. For example, Colorado uses several risk adjustment models from 3M™ Health Information Systems, including:

• All Patient Refined DRGs (diagnosis-related groups),
• Enhanced Ambulatory Patient Grouping System for health care services, and
• Clinical Risk Grouping Software for population-based comparisons.⁷⁹

Using these risk adjustment methods has allowed the Colorado APCD to determine an average illness burden score to compare utilization and cost across patient populations, such as at the county level.

Finally, measures need to provide actionable information and a framework for improvement to realize substantial gains in quality, cost, utilization, efficiency, or population health.⁷⁴ For example, process measures capture the percentage of time providers conform to standards of clinical care. Public reporting may increase adherence to national guidelines and, in turn, improve health-related outcomes in patients. However, adherence requires timely feedback to clinicians so that they can change behaviors in real time.

In addition, providers need to be invited into the data auditing process and offered:

• Notice that new scores are forthcoming,
• An explanation of methodology and data,
• Opportunities to review the data and appeal the results of performance analyses, and
• An independent external review process to encourage provider participation and action based on the measures.⁶⁸

Issue 4: Measure Gaps

Limited discussion was found in the literature and in the environmental scan regarding measurement gaps related to the scope of this project. While examples of gaps identified during our review are provided here, this is not a complete list of existing methodological gaps or gaps in available measures.

Measures validated by NQF and other national organizations are widely available, and most States rely on such measures as the basis for their measurement programs. However, in some areas, few validated measures exist or those that exist are inadequate or proprietary, making identification of useful, accurate, and meaningful measures to implement a challenge.

Cost measures used, particularly those used for State price transparency Web sites, vary substantially, and many measures may not have been validated yet. Since it is critical to ensure that valid cost measures are used to avoid the unintended consequence of physicians avoiding high-cost patients and causing disparities in care, this area needs much work.⁴⁸ For example, there is considerable variation in the reporting of cost at the State level. For cost estimates given by New Hampshire's price transparency Web site (http://nhhealthcost.nh.gov), costs are based on the median allowed rate by insurers and the median out-of-pocket costs for patients. On the other hand, Virginia’s price transparency Web site (http://www.vhi.org/healthcarepricing/) reports the average allowed amounts for services.

Most Web sites do not display cost and quality data in the same place, making it difficult for users to assess value. Some cost measurement Web sites have also used episode groupers to calculate costs, but these measures are limited in that they report costs for episodes as though they occur in patients without comorbidities. A more "person-centric" approach has been called for, but the steps to implementation were not described in the resources reviewed for this report.

Finally, in terms of implementation, States need to consider how best to display data, as data display can have a significant impact on consumer understanding and use of the reports.⁴¹ States should also develop criteria for redacting measures and excluding them from publications, if warranted.

Methods for calculating patient out-of-pocket costs vary as well, as the amount depends on the benefit design structure. Only two States, New Hampshire and Colorado, attempt to estimate out-of-pocket costs for patients. Most consumers have access to insurer sites, but the information on those sites is proprietary, so consumers would not be able to shop for prices between plans, only within a plan.

In considering how to fill the out-of-pocket costs measure gap, a useful resource is the Pacific Business Group on Health's summary sheet of the structures of consumer-facing cost measures offered on the major insurers’ proprietary Web sites (PBGH Consumer Decision Support Tools). Reviewing the existing options for how to develop and present these measures would allow a deeper discussion about the relative merits of one approach over another.

Provider-level measures are another gap that needs to be addressed. As discussed above in the section discussing difficulties with data linkage and aggregation, provider attribution is challenging, and providers may not care for enough patients with a given condition to reliably assess quality of care. Establishing a Master Provider Index may help address this issue. Furthermore, APCDs do not contain data to measure health status or outcomes, so calculating value purely from APCD data may be very difficult.

Finally, methodological gaps also require attention as States increasingly use APCDs for measurement, particularly in terms of evaluating resource utilization and cost. For example, it is not clear how to account for complications of care, particularly when discussing chronic disease care. Does an acute myocardial infarction count as a new episode of care or does it count toward the year’s expenditures for management of coronary artery disease?

Accounting for total utilization also may be a challenge since different services have different levels of intensity (e.g., lab test versus surgery) and different geographic areas have different prices. Creating a weighted sum of services, for example, could help overcome this barrier.⁶ Each service can be weighted by a standard national price for that service to remove the effect of different prices across geographic locations and derive an aggregate measure of services. Of course, this approach is only desirable for utilization analyses. If the focus is on cost of care, then both utilization and amounts paid are relevant, so one would not want to use a standard price list.

The above measurement gaps are only examples of the significant challenges States face when trying to use APCDs to measure health care cost, quality, and utilization. We anticipate that the detailed measure review undertaken during measure inventory development will help reveal additional measure gaps and information to assist policymakers, measure developers, researchers, and others in filling those gaps.

While considering these barriers, it is useful to note that as APCDs become more developed and the data become more detailed (e.g., with the creation of universal provider directories), some use cases may become more common. A recent APCD Council report describes the staged or tiered approach several States have taken to releasing analytic products, starting with the release of statewide or county-level measures before the release of more granular analysis (e.g., by provider or payer).⁵³ This approach was also described by TEP members in the February 2015 meeting. The approach allows early public reporting of policy-relevant data from APCDs for State decisionmakers, while stakeholders address the technical and political barriers described above to allow more granular reporting.

ⁱⁱⁱ The 14 States are Colorado, Connecticut, Maine, Maryland, Massachusetts, Minnesota, New Hampshire, Oregon, Pennsylvania, Rhode Island, Tennessee, Utah, Vermont, and Virginia.

Next Page