All-Payer Claims Databases Measurement of Care: Systematic Review and Environmental Scan of Current Practices and Evidence

The methodology for identifying empirical literature in this review involved two primary mechanisms. In the first mechanism, d structured search strategies were performed run October 13, 2014, and October 27, 2014, with an additional search term as described below. The search included PubMed, EconLit, Embase^®, and Web of Science, as well as the first three pages of a search from Google Scholar, sorted by relevance, and date restricted (2008-2014).

The search included articles since January 2008, when the first all-payer claims databases (APCDs) were formed. These searches provided the initial capture of references to articles on the use of claims data for measurement. The searches included studies that were empirical in nature and overview-type publications (e.g., commentaries, editorials, or viewpoints).

The overview publications were included to help shape the guiding questions for which there is more limited empirical data:

• Measure gaps and concepts proposed to fill those gaps (question 3),
• Potential barriers to using and reporting measures with APCD data and strategies suggested to overcome those barriers (question 4), and
• Methodological considerations in using APCD data for measurement (question 5).

Some issues that pertain to APCD use occur with any claims database with more than one payer and literature includes multi-payer claims databases (MPCDs) in addition to the limited number of APCDs that have been developed. Therefore, the search included MPCDs, so as not to miss important findings from MPCD experiences that would be relevant to APCDs.

The search strategies were limited to the type of data and the approach used in the paper (e.g., “all-payer,” “cost,” “episode of care”) rather than specific clinical terms or titles of quality measures that might be needed for a more focused topical review of quality measures using APCDs. This approach was chosen to avoid limiting the review to specific clinical areas or areas of quality measurement and to avoid capturing a large number of empirical studies of quality measures that cannot be used with APCDs or other claims databases.

Studies were also excluded that relied solely on Healthcare Cost & Utilization Program (HCUP) data. In the preliminary search, studies using the following HCUP databases were reviewed:

• National (Nationwide) Inpatient Sample (NIS),
• Kids Inpatient Database (KID),
• Nationwide Emergency Department Sample (NEDS),
• State Inpatient Databases (SID),
• State Emergency Department Databases (SEDD), and
• State Ambulatory Surgery and Services Databases (SASD).

These databases could not be used for episode-based and longitudinal measures that cross settings.

In consultation with the AHRQ task order officer (TOO), the project team excluded sources that rely exclusively on HCUP data, because AHRQ is well informed of the strengths of the HCUP databases and the visit-level measures that have been developed using them. Focusing on sources that propose or implement measurement using MPCDs or APCDs allows us to explore this innovative literature in greater depth.

Lastly, while reviewing the first search and during discussions with the TOO, the team decided to modify the search strategy in two ways. They decided to include in the set of first search terms “claims data” without restricting it to “all payer” or “all-payer.” This decision was made because the volume of articles found was low when restricted to “all payer,” “all-payer,” or APCD, and it was difficult to find any empirical studies using MPCDs.

In addition, the team decided to include “episode” in the list of second search terms (see “Search Strategy Details” below for a description of how the searches were built). Being able to create episodes of care, which follow a patient across settings (inpatient and outpatient) and over time, is an innovative aspect of the claims databases. This innovative aspect may be the basis for a rationale to develop and maintain an APCD, so the team decided to optimize the search to capture these types of empirical studies.

In the second mechanism to capture literature, other experts on the core team and the TOO were asked to provide recommendations on seminal work in APCD measurement, empirical research using APCD measures, and guidance regarding any additional scope of work that should considered.

To inform this review, the team used some secondary or “chain-method” capture of references in reviewing key resources. That is, reference lists in articles captured from the first two mechanisms provided additional capture of literature. However, given the brief nature of the current review, the team used this method only for seminal papers, rather than all research and overview papers.

Abstracts of all articles were reviewed and included if they were within the scope of the guiding questions. Figure 1 at the end of this appendix displays the results from identification, screening, and selection of literature, including number of articles from each search and reasons for exclusion.

Return to Contents

Search Strategy Details

The same search strategy was used for each database in the following order, excluding duplicates found in an earlier database: PubMed, EconLit, Embase, and Web of Science, and the first three pages of a search from Google Scholar. The Google Scholar search engine has a sidebar that allows a custom date range and an option to sort by relevance or by date. The team restricted the date (2008-2014) and sorted by relevance.

The two searches captured literature indexed on or before October 13, 2014, and subsequently on October 27, 2014. Each database was searched using the following phrases in the free-text search engines in each database, without restricting the search to a specific search field or fields. Searches had one of the following terms as a first term: “all-payer” OR “all payer” OR “APCD” OR “all-payer claims data” OR “claims data.”

With the following terms as a second term, the team searched in the following order:

Quotation marks around the phrases were not used in any of the searches.

Return to Contents

Abstraction

Each paper title and abstract was reviewed to determine whether:

• The study was within the scope of one of the guiding questions,
• It was within the date range chosen,
• It included data that were not only inpatient data,
• It included non-claims data, and
• It was literature supporting a measure that was already endorsed nationally (and thus would form the basis for a measure in the environmental scan).

Seventy-six papers were excluded during this phase per those criteria. The team included for a full paper review those for which the above criteria could not be fully determined based on the title and abstract.

A total of 128 full papers were reviewed. An additional 30 papers were excluded that were beyond the scope of the review or had measures in the empirical study that would have formed the basis for an APCD measure but also required non-claims data (e.g., survey data or clinical data not available in the claims database).

For the papers reviewed in full, the following data elements were abstracted: type of paper (overview or empirical research paper); type of data (APCD, MPCD, or other claims, such as Medicare data); type of measure (cost, quality, utilization, or multiple); and unit of analysis (population based, comparative—provider, health plan, geographic). At this point, measurement specifications were not abstracted for each paper.

Figure 1. Literature Identification, Screening, and Selection Process

Next Page