Care Coordination Measures Atlas Update

Main Indicator Sources

1. Literature Search. A measure-specific care coordination search was conducted to identify published literature related to the development, validation, and testing of measures of care coordination. The search strategy is outlined below.
2. Care Coordination EPC Report. As part of a previously published care coordination report ("Closing the Quality Gap: A Critical Appraisal of Quality Improvement Strategies"; Volume 7: Care Coordination), background research and a systematic review identified care coordination indicators in published studies.
3. Panelist Calls. A series of panel calls were held in order to obtain information regarding additional measures of care coordination and ongoing research and development in the field. Panel participants had backgrounds ranging from research and evaluation in care coordination to clinical practice. For a list of panel participants, please go to Appendix III: Advisory Group Participants.
4. NQF Draft Report. NQF evaluated a list of 77 candidate measures and recommended a set of preferred practices across five domains of care coordination: 1) health care home, 2) proactive care plan, 3) communication, 4) information systems, and 5) transitions. The final report was released in October, 2010, shortly before completion of the Atlas.¹⁶

Literature Review Search Strategy

The final measure search was performed on July 13, 2010, using the following strategy:

[ ("(("healthcare" or "healthcare" or care) adj3 (coordinat* or "co-ordinat*" or integrat*)).tw.") And ("(rated or rating or indicator* or measure* or valid* or reliab* or outcome* or model* or scale* or subscale* or questionnaire*).tw. or methods.fs. or exp Questionnaires/") ] Not [ ("exp geographic locations/ not exp united states/") ]

The search was limited to English language publications. Details of the search strategy development are included in the box below.

In addition, publications by known key researchers involved in care coordination measurement were also searched. Bibliographies of particularly relevant included references were also reviewed for any further sources of information.

We compared our search strategy to RAND's ACOVE-3 search strategy post-hoc, and we found no additional terms, phrases, or combinations that were not captured in the strategy outline above.

Measure Selection

Measures for inclusion in the Atlas were identified in two steps. First, a list of potential measures was compiled from the search strategies outlined above, which yielded a total of 3448 unique measure sources. Measure sources were included if they featured any relation to measurement or evaluation of care coordination or of any of the care coordination domains included within our measurement framework, with an emphasis on specific instruments or measures. Although the ambulatory setting is the focus of this project, we did not exclude sources discussing measurement of care coordination in nonambulatory settings. Validity, testing, or feasibility of measures were not considered during this review phase.

A single reviewer compiled the potential measures list after reviewing titles and abstracts of all search results. A second reviewer provided input on measure sources for which inclusion was unclear and a decision was made through discussion.

Based on this preliminary review, 149 potential measure sources were identified to consider for inclusion in the Atlas. It is important to note that in some cases multiple sources related to a single measure, and in other cases a single source discussed multiple measures.

In the second step, we reviewed the full text of all articles on the potential measure list and made decisions about whether they should be included within the Atlas. Measures were excluded if, in the opinion of the reviewer, they did not meet all of the following criteria:

1. Clear relevance to care coordination or at least one of the care coordination measurement framework domains. Measures that did not include at least one instrument item that mapped to at least one framework domain were not included. Measures that contained only 1 or 2 minimally relevant items within a large instrument unrelated to care coordination were also excluded.
2. A clearly defined and reproducible measure yielding quantitative data. Examples of evaluations that did not meet this criterion were interview guides, focus group reports, or free-response questionnaires yielding textual data that required content analysis; quality improvement guides designed to walk users through a process of self-evaluation without yielding measurable data; and evaluations of specific programs or interventions tailored to the subject of study in such a way as to make use in any other situation very difficult without major modification.
3. Information available demonstrating some valid measurement properties or that the measure was developed in association with a logic model that has evidence of causal linkages between the activities measured and outcomes desired. Measures that underwent testing and were shown to have poor validity or reliability were not included in the Atlas.

In many cases, additional sources were consulted to address the testing criteria. When the decision about whether to include a measure seemed unclear, the primary reviewer consulted with additional team members and a decision was made through discussion.

Of the 149 potential measure sources identified, 70 were excluded. Of these, 31 were excluded due to lack of relevance (criterion 1); 34 were excluded because they were not a clearly defined, quantitative measure (criterion 2); and 38 were excluded due to unknown or poor validation or testing (criterion 3). Thirty-five potential measure sources met more than one exclusion criteria. In addition, 7 measures sources were excluded because we could not identify information necessary to assess suitability for inclusion in the Atlas. Our attempts to obtain the missing information from the developers of these measures were unsuccessful at the time of publication. A further 18 measure sources were not unique; that is, multiple sources pertained to a single measure. These sources were used to create the profiles but did not themselves contribute a unique measure.

In all, we include 61 measures in the Atlas, which are detailed in 78 profiles. The number of profiles is greater than the number of measures because for measures with multiple versions, we created separate profiles for versions with substantially different question items. In instances where the only difference between versions was a minor wording change to reflect a different population, setting, or year, we created just one profile to represent all versions.

Atlas Update Measure Selection

A similar process of measure selection was utilized during the Atlas update search. In total, we screened 1346 articles from the peer reviewed literature. Of these, we reviewed the full text for 157 articles, of which 19 yielded at least one potential measure. In addition, we reviewed a further 57 secondary sources, such as articles included within a systematic review. Together, these sources yielded 32 potential new measures that met the Atlas inclusion criteria. A further 133 potential measures were identified through the environmental scan, of which 65 were eligible for inclusion in the Atlas. The same Atlas measure selection criteria were used, with the following modifications:

• Measures that were currently endorsed by NQF at the time of review, or that were based on evidence-based guidelines, were considered to meet the validity criterion.
• Only measures applicable to the primary care setting were included in this update. This includes measures that are not setting specific, or that are applicable in primary care as well as other settings. Primary care was selected as a focus given its often central role in coordinating care across settings, particularly as accountable care and patient-centered medical home delivery models are more widely implemented. Furthermore, this focus aligns with the original scope of the Atlas that centered on measures that might reasonably be applied in the ambulatory care setting.
• Measures tailored towards individuals with a particular disease or condition were not included in the update. This exclusion reflects the desire to focus on measures that are broadly applicable. Measures applicable to patients with a range of chronic diseases were included, but those applicable to only a particular disease were not.

In addition, measures relying exclusively on chart review or administrative claims data (i.e., not utilizing survey methods) were not included in the Atlas update. However, EHR-based measures are reviewed in the chapter on emerging trends in care coordination measurement, although individual profiles for these measures are not included.

In total, the Atlas update includes 20 additional measures in 22 profiles. A further 26 EHR-based measures are reviewed in Chapter 4.

Limitations

Although we attempted to identify as many potential measures of care coordination as possible through our various search strategies, we relied primarily on published instruments available in the public domain. Instruments not published in journals were identified chiefly through suggestions from our review panels. This method omits an unknown number of potential measures that were not published in the literature, not identified by our search terms, or were not recommended for review by our advisory panels. When a potential measure of care coordination was reported in the literature without including the measure instrument, we contacted the article author to request a copy of it. We were also limited in our ability to provide information on the feasibility and cost of using measures by what was reported in the literature; few studies describe these aspects of measurement.

¹⁶National Quality Forum. Preferred practices and performance measures for measuring and reporting care coordination: a consensus report. Washington, DC: National Quality Forum; 2010.

Return to Jump Start