Measure Mapping Procedures
Before beginning the mapping process, the research team developed domain definitions and the Measure Mapping Table (go to Table 6).
To begin the mapping process, all measures included in the Atlas were reviewed by one of two reviewers. Reviewers read through each measure, keeping in mind the specific components of care coordination that the measure addresses. Using the measure mapping table and the domain definitions, the reviewers identified the specific domains that correspond to the components of care coordination that each instrument measures. Reviewers also determined the perspective of measurement. When questions arose about appropriate mapping, the reviewers discussed and ultimately agreed upon a perspective and set of domains corresponding to each measure.
To assess inter-rater reliability of the measure mapping, reviewers selected at random 6 measures (approximately 10 percent of the total included in the Atlas) from among 31 measures that were planned for inclusion within the Atlas at the time the reliability testing was performed. The 19 measures included in an earlier draft Atlas were not considered for reliability testing because their mapping was discussed during development of the draft. The reviewers also did not consider reliability testing on those measures for which a final decision had not yet been made about whether it would be included in the Atlas, or measures that were missing key information (such as the measure instrument) at the time of reliability testing. Three measures were selected randomly from among those mapped by reviewer 1 (n=9 measures total) and three were selected randomly from among those mapped by reviewer 2 (n=22). Reliability was assessed before any discussion among the reviewers regarding the selected measures.
Across the 6 measures, there were 169 individual measure items (e.g., survey questions). Agreement about whether a specific item mapped to any domain was 86 percent (146/169), with a kappa of 0.694 (p<0.001). Conventionally, a kappa >0.67 is considered sufficient for drawing some conclusions. Therefore, we believe that the observed kappa of 0.69 is sufficient for the purposes of the measure mapping, which is intended to facilitate identification of relevant measures.
We also assessed reliability of mapping to the 3 perspectives: patient/family, health care professional(s), and system representative(s). Across 6 measures and 3 perspectives, there were 18 possible perspective mappings. (Each measure may be mapped to multiple perspectives). Reviewers agreed on all but one combination, resulting in 94 percent agreement. We did not calculate a kappa statistic because it is not an appropriate statistic when more than one mapping is possible for each measure.
Reliability of mapping to the framework domains was also assessed. To assess agreement of domain mapping across measurement items, only items that were mapped by both reviewers (n=101) were considered. Subdomains (e.g., Interpersonal Communication and Information Transfer) were considered as distinct domains for the purposes of reliability assessment.
Domain mapping agreement was examined in two ways. First, we examined agreement by domain. That is, what proportion of the 101 measure items did both reviewers agree should be mapped to each domain? Agreement in mapping to domains was good, ranging from 80 percent (Communicate) to 100 percent (Facilitate Transitions as Coordination Needs Change; Health Care Home; Health IT-Enabled Coordination).
Reliability of domain mapping was also assessed by comparing mapping across measure items. That is, how similar were each reviewer's mappings for each item? For this comparison, the denominator was calculated by multiplying the total number of items mapped (n=101) by the total number of possible mappings (17 domains). Agreement was excellent. The reviewers agreed on 1604/1717 possible mappings, or 93 percent. As was the case for the perspective reliability assessment, a kappa statistic was not calculated because it is not an appropriate statistic when more than one mapping is possible for each measure.
Measures added to the Atlas as part of this update were mapped in the same manner as outlined above for the original Atlas. All mapping was confirmed by a member of the original Atlas development team, with an emphasis on consistency in how domains were applied during mapping.
Examples of Measure Item Mappings
The following list provides sample items (and their measure source) that were mapped to each care coordination domain on the measure mapping table. Copies of the measure instruments will be added to Appendix IV: Care Coordination Measures. Appendix IV will be updated regularly.
Establish Accountability or Negotiate Responsibility
- I clarify whether the nurse or I will have the responsibility for discussing different kinds of information with the patient. [Measure #7b, item 10 (CPS)]
- How often were you confused about the roles of different providers? [Measure #6. item 9 (CPCQ)]
Communicate*
- Across health care teams or settings How effective is one-to-one communication between ICU staff and members of other units? [Measure #12a. item VIIB.f (ICU Nurse-Physician Questionnaire)]
Interpersonal Communication
- Between health care professional(s) and patients/family How often does your service provider talk with you about your future care? [Measure #6, item 27 (CPCQ)]
- Within teams of healthcare professionals I discuss areas of agreement and disagreement with nurses in an effort to develop mutually agreeable health goals. [Measure #7b, item 5 (CPS)]
Information Transfer
- Across health care teams or settings Medical record transfer: IF a person age 75 or older is transferred between emergency rooms or between acute care facilities, THEN the medical record at the receiving facility should include medical records from the transferring facility, or should acknowledge transfer of such medical records. [Measure #2, item 11 (ACOVE-2 Quality Indicators)]
- Within teams of health care professionals It is often necessary for me to go back and check the accuracy of information I have received from nurses in this unit. [Measure #12b, item I-4 (ICU Nurse-Physician Questionnaire)]
Facilitate Transitions†
Across Settings
- Did your primary care provider (PCP) or someone working with your PCP help you make the appointment for that visit (referred to specialist)? [Measure #17a, item E9 (Primary Care Assessment Tool-Child Edition (PCAT-CE))]
As Coordination Needs Change
- In preparation for transition (to adulthood), does your provider have a process to share information with the adult care provider including: transition plans, medical records, key health issues, and current family and youth roles in managing care? [Measure #11a, item 4.2E (FCCSAT-Family Version)]
Assess Needs and Goals
- Before I left the hospital, the staff and I agreed about clear health goals for me and how these would be reached. (Y/N) [Measure #9b, item 1 (CTM-15)]
Create a Proactive Plan of Care
- When I left the hospital, I had a readable and easily understood written plan that described how all of my health care needs were going to be met. [Measure #9b, item 1 (CTM-15)]
Monitor, Follow Up, and Respond to Change
- In the past 3 months, how often have service providers responded appropriately to changes in your needs? [Measure #6, item 10 (CPCQ)]
- Diagnostic test followup: IF the outpatient medical record documents that a diagnostic test was ordered for a person age 75 or older, THEN the medical record at the followup visit should document 1 of the following: result of the test, test was not needed or reason why it will not be performed, test is still pending. [Measure #2, item 6 (ACOVE-2 Quality Indicators)]
- Does your partnership with your provider change over time as your experiences, knowledge, and skills change? [Measure #11a, item 1.8 (FCCSAT-Family Version)
Support Self-Management Goals
- When I left the hospital, I clearly understood the warning signs and symptoms I should watch for to monitor my health. (Y/N) [Measure #9b, item 6 (CTM-15)]
- In the past 3 months, how often did someone on your diabetes care team teach you how to take care of your diabetes? [Measure #21, item 7 (RSSM)]
Link to Community Resources
- Linking patients to outside resources: 1) is not done systematically; 2) is limited to a list of identified community resources in an accessible format; 3) is accomplished through a designated staff person or resource responsible for ensuring providers and patients make maximum use of community resources; or 4) is accomplished through active coordination between the health system, community service agencies, and patients. [Measure #1, item 7 (ACIC)]
Align Resources With Patient and Population Needs
- Do you and your staff: Offer trained interpretation (foreign language or sign)? [Measure #11b, item 13.1C (FCCSAT-Provider Version)]
- Is your facility able to change health care services or programs in response to specific health problems in the communities? [Measure #17c, item J4 (PCAT-FE)]
Teamwork Focused on Coordination
- When problems arise regarding the care of ____ patients, do care providers in these groups work with you to solve the problem? [Measure #46, item 4 (RCS)]
- Overall, our unit functions very well together as a team. [Measure #12a, item V.9 (ICU Nurse-Physician Questionnaire)]
Health Care Home
- Is there a doctor or place that you usually take your child if s/he is sick or you need advice about his/her health? [Measure #17b, item A1 (PCAT-AE)]
Care Management
- Does anyone help you or coordinate [CHILD'S NAME]'s care among the different doctors or services [he/she] uses? (asked for children who used more than two services) [Measure #51, item K5Q20 (NSCH)]
Medication Management
- The pharmacist and I negotiate to come to an agreement on our activities in managing drug therapy. (Y/N) [Measure #18, item 7 (PPCI)]
Health IT-Enabled Coordination
- What is the policy timeframe for clinicians to respond to patient PHR emails?14 [Measure #34, item 10 (PHR)
Additional Measure Characteristics
In this updated version of the Atlas, all measures are characterized with respect to three additional criteria: patient age groups, patient conditions, and settings. These criteria identify the group or groups of patients whose care the measure is intended to assess. They are not applicable to all measures. For example, some measures focus on aspects of coordination as it is performed or perceived by health care professionals and patients are not directly addressed. The Setting categories identify the settings for which the measure is designed or where it is intended to be or has been used. Measures were mapped to these categories based upon information contained in the measure instrument itself and in published sources listed in the Atlas profiles. Measures were mapped to a category if it matched a stated intent or purpose of the measure or a published use of the measure. When possible, feedback from measure developers was incorporated prior to finalizing the categorization for each measure. Definitions for categories within each of these criteria are listed below.
Patient Age Group
The Patient Age Group criterion identifies the group or groups of patients whose care the measure is intended to assess. This criterion is based upon use of the measure in sources listed in the Atlas profiles or information contained within the measure instrument. Categories are not mutually exclusive. For example, measures that are classified as Older Adults are also classified as Adults. Similarly, measures that have been used in both adult and pediatric populations are classified as both Adults and Children.
Children – Measure is targeted toward or has been used in a patient population described as pediatric, children, or parents/care takers of children receiving health care.
Adults – Measure is targeted toward or has been used in an adult population. This includes measures applicable to older adults.
Older Adults – Measure is targeted toward or has been used in older adults, including measures designated for a geriatric patient population, the elderly, or aged individuals. All Older Adult measures are also included in the Adult category.
Not Age Specific – Purpose states measure is intended for application to patients of all ages, or no information is available on the ages of patients to whom the measure has been applied.
Not Applicable – Measure does not focus on patients.
Patient Condition Group
The Patient Condition Group is the disease or condition for which the measure is targeted and the population that requires or benefits from coordination. This criterion is based upon use of the measure in sources listed in the Atlas profiles or information contained within the measure instrument. Categories are not mutually exclusive. Measures are mapped to more than one condition category when applicable.
Combined Chronic Conditions - Patients with any chronic condition, including patients with conditions captured by one of the other Patient Condition Group categories. This category includes all measures mapped to General Chronic Conditions, Multiple Chronic Conditions, Cancer/Oncology, Mental Illness & Substance Use Disorders, and Children with Special Health Care Needs.
General Chronic Conditions - We include in this category patients who are described as having chronic conditions, chronic diseases, or chronic illnesses without specifying particular conditions. We also include in this category any specific chronic disease that is not captured by one of the other patient condition categories. We define chronic condition as a disease or condition of long duration and typically slow progression. We do not restrict the definition of chronic condition to specific diseases, but the following are examples of conditions that would be included: HIV/AIDS, asthma, chronic obstructive pulmonary disease, diabetes and cardiac conditions, including congestive heart failure and coronary artery disease. Measures included in the Multiple Chronic Conditions category are also included here.
Multiple Chronic Conditions - Patients with at least two simultaneous chronic conditions. These may be two or more specific chronic diseases (e.g., congestive heart failure and diabetes), or a description of patients as having multiple chronic conditions, diseases or illnesses without specifying particular conditions. All measures included in this category are also included in the Combined Chronic Conditions category. If applicable, measures included here may also be included in one of the other Patient Condition Group categories (e.g., a measure designed for patients with diabetes and mental illness is also included in the Mental Illness & Substance Use Disorders category).
Cancer/Oncology - Patients with any form of cancer, including leukemia, or patients of any oncology service or provider. This category also includes patients who are undergoing diagnosis for cancer because coordination issues during the period of diagnosis are likely similar to those during the treatment phase. All measures included in this category are also included in the Combined Chronic Conditions category.
Mental Illness & Substance Use Disorders - Patients with any mental illness, such as depression, schizophrenia, bipolar disorder, obsessive compulsive disorder, anxiety disorders, and post-traumatic stress disorder. This category also includes alcohol or substance abuse and unspecified mental illness or mental disorders. All measures included in this category are also included in the Combined Chronic Conditions category.
Children with Special Health Care Needs - Children who have or are at an increased risk for a chronic physical, developmental, behavioral, or emotional condition and who also require health and related services of a type or amount beyond that required by children generally. All measures included in this category are also included in the Combined Chronic Conditions category.
Other Conditions - Measure is targeted toward patients with a specific condition not captured by one of the above categories.
General Population or Not Condition Specific - Measure is targeted toward or has been applied to the general population or to a patient group not limited by condition. Validation or application of the measure is not limited to particular patient disease or condition groups, or the disease/condition of interest was not specified.
Not Applicable - Measure does not focus on patients.
Setting
The Setting is the location where the measurement is taking place. Categories are not mutually exclusive. Measures may map to more than one category.
Measures that focus on a particular transition point are mapped to both the before and after setting (e.g., a measure of the transition from hospital discharge to outpatient care would map to Inpatient Facility and Primary Care Facility). Transitions from hospital to home are mapped to Inpatient Facility and Primary Care Facility, since patients discharged home are typically expected to follow-up with their primary care provider. When the setting is specified as “ambulatory care” without more specific details, the measure is mapped to Primary Care Facility and Other Outpatient Specialty Care Facility.
When a specialty is mentioned but it is not specified whether the specialist consult, visit or services were delivered through inpatient or outpatient settings, the measure is categorized as Not Setting Specific. For particular measures, some applications labeled the setting while others did not; these were mapped to both a specific setting (e.g., Primary Care Facility) and Not Setting Specific. Not Setting Specific is only checked if there is no setting specified in at least one application of the measure.
Inpatient Care Facility - Any care received by a patient admitted to any department of an acute care hospital. This includes patients admitted to the psychiatric department of acute care hospitals. This category does not include services provided by hospitals without an admission (e.g., emergency department, outpatient clinic or same-day surgery).
Emergency Care Facility - Care provided in an emergency department, ER, urgent care clinic, or other emergency setting, with or without an admission.
Primary Care Facility - We define ambulatory primary care facility as any setting described as primary care, or settings providing care by generalists or practitioners in internal medicine, family practitioners, general pediatricians or general practice providers. This includes settings described as a medical or healthcare home or PCMH.
Other Outpatient Specialty Care Facility - Any outpatient care facility that does not meet the definition of any other setting category. This includes outpatient specialty clinics such as cardiology, orthopedics, and also outpatient or same-day surgery centers. It does not include outpatient mental or behavioral health centers (classified as Behavioral Health Care Facility) or urgent care clinics (classified as Emergency Care Facility).
Behavioral Health Care Facility - Care provided in any facility or setting that specializes in mental or behavioral health. This includes psychiatric hospitals, substance use treatment centers and behavioral health clinics. It does not include the psychiatric department of an acute care hospital (classified as Inpatient Care Facility).
Long Term Care Facility - Any long-term care facility or institutional care setting, including nursing homes, skilled nursing facilities, incremental care facilities for the mentally retarded, residential care settings, or step-down facilities.
Home Health Care - Health or supportive care provided in the patient’s home by health care professionals.
Other Setting - Any other setting not included in one of the above categories, but that is specifically noted as the target location for use of the measure, or where the measure has been used in the past (in published work). This might include coordination around physical or occupational therapy, rehabilitation, etc.
Not Setting Specific - The measure application is not limited to a particular type of setting, or the setting was not specified in measure development or application publications.
*Note 1: When the mode of communication was not clear, measures and measure items were mapped to the less specific Communicate domain rather than to either of the subdomains (Interpersonal Communication and Information Transfer).
†Note 2: We were able to map all measures related to transitions to one or the other of the subdomains specifying transition type (Facilitate Transitions Across Settings and Facilitate Transitions as Coordination Needs Change). Therefore, no measures or measure items were mapped to the less specific Facilitate Transitions domain.
