HCOs are resource constrained, and efforts to measure quality and safety of healthcare are anchored foremost to the measures specifically required by accrediting agencies and payers. Currently, measures of diagnostic performance and safety are not among these required measures, and the burden of additional data collection is a daunting prospect for HCOs that already struggle to meet existing requirements. Furthermore, we need national consensus regarding what aspects of diagnostic safety can be measured pragmatically in real-world care settings.
Despite the additional burden that measurement for improvement would entail, the benefit of these activities for improving patient safety and stimulating learning and feedback could outweigh concerns.
The current lack of incentives and models to measure diagnostic safety can make it difficult to get started even in the best case scenario when leadership support and resources are available to support improvement efforts focused on diagnosis. Nevertheless, as the burden of diagnostic errors is increasingly recognized and as measurement strategies become better defined, diagnostic safety will no longer remain sidelined in the landscape of healthcare quality and safety.
Despite the additional burden that measurement for improvement would entail, the benefit of these activities for improving patient safety and stimulating learning and feedback could outweigh concerns. In fact, some HCOs have already started on their journeys, including one that has pursued measurement on multiple fronts103 and another that aims to pursue measurement as part of becoming a “Learning and Exploration of Diagnostic Excellence (LEDE)” organization.33 Now is the time for other HCOs to use the data already available to them to begin to detect, understand, and learn from diagnostic errors.
While diagnostic errors occur across the spectrum of medical practice, measurement should be strategic and focused on areas with strong potential for learning and impact.
While diagnostic errors occur across the spectrum of medical practice, measurement should be strategic and focused on areas with strong potential for learning and impact. As with other aspects of patient safety measurement, goals should include:
- Creating learning opportunities from past events with both potential and real harm.
- Ensuring reliability in diagnostic safety.
- Anticipating and preparing for problems related to diagnosis.
- Integrating and learning from the knowledge generated.13
Absent a specific local or institutional need or mandate calling for a specific measurement target, we recommend that HCOs new to measurement of diagnostic safety focus initial measurement efforts on a limited set of realistic measures that map to one or a few specific diagnoses or care processes. Measurement targets may be in line with:
- Published literature (e.g., research showing a high rate of missed test results).
- Priorities identified in national initiatives (e.g., timely diagnosis of cancer), or
- Local needs identified by quality and safety committees (e.g., focusing on specific symptoms such as abdominal pain104 or specific diseases such as spinal epidural abscess).70
Once a target is identified, HCOs should use measurement strategies that balance validity (i.e., for case finding) with yield. Figure 2 visualizes the implementation readiness of several diagnostic safety measurement strategies discussed in this brief. For example, e-trigger enhanced structured chart review appears suitable for operational measurement with additional development. Systems without EHR capabilities can use other data sources (e.g., selective reviews, event reports) to begin measurement as soon as possible, rather than wait for the information technology infrastructure needed to implement EHR-based algorithms. The strategies depicted in Figure 2 represent the current state of the science and are subject to change as new innovations are validated.
Figure 2. Implementation Readiness of Diagnostic Safety Measurement Strategies
Note: Larger circles denote higher potential yield for cases that can inform systemwide learning and improvement. Measurement strategies that are ready for implementation balance validity and yield (i.e., an estimate of the proportion of cases with diagnostic errors that could lead to learning and improvement relative to measurement effort). The relative position of these methods will vary according to local context; Figure 2 illustrates data sources for a typical healthcare organization equipped with an EHR and moderate resources for quality and safety improvement.
Table 3 presents a summary of the strategies and the factors affecting their use.
Table 3. Implementation Readiness of Diagnostic Safety Measurement Strategies and Estimated Yield Relative to Effort
| Measurement Strategy | Stage of Development |
Current Potential Availability and / or Accessibility of Data Source |
Estimated Yield Relative to Effort |
|---|---|---|---|
| Review of solicited reports from patients | Exploratory | Low | Medium |
| Advanced data science methods using EHR data (e.g., NLP) | Exploratory | Low | Very large |
| Mining administrative billing data | Exploratory | High | Very small |
| E-trigger enhanced chart review | Moderate | Moderate | Very large |
| Institutional peer review processes | Moderate | High | Medium |
| Morbidity and mortality conferences | Moderate | High | Medium |
| Review of solicited brief reports from clinicians | Moderate | Moderate | Very large |
| Selective chart review of high-risk cohorts | Mature | High | Large |
| Random chart review | Mature | High | Very small |
| Review of autopsy reports | Mature | Low | Large |
| Review of malpractice claims | Mature | High | Medium |
| Review of incident reports | Mature | High | Small |
Based on what HCOs are learning, they could conduct additional activities related to quality improvement, such as:
- Better managing test results to make sure they are acted on.105
- Conducting a self-assessment of reliability of communication and reporting of test results.106
- Closing referral loops.107
- Enhancing teamwork and communication with patients.108
Meanwhile, research should stimulate the development of more rigorous measures that are better correlated with not just more timely and accurate diagnosis but also less preventable diagnostic harm.
For additional implementation and adoption, especially beyond measurement to inform institutional quality improvement activities that we discuss in this brief, additional balance measures will need to be developed. For example, measures that focus on underdiagnosis of a certain condition (stroke) may lead to overtesting for that condition (magnetic resonance imaging on all patients with dizziness in primary or emergent care). Thus, research efforts must continue to inform and strengthen the rigor of measurement.
Other unintended consequences, including gaming and measure fatigue, may emerge from implementing suboptimal and inadequately validated performance measures for public reporting, performance incentives, or penalties. For measures to be used for accountability, they will first need to be tested and validated and linked to improving suboptimal care or outcomes.109 This document and its recommendations thus focus largely on measurement for improvement at the HCO level and not measurement for accountability purposes.
In the long term, measures will need to be developed to address diagnostic excellence, defined in terms of the ability to make a diagnosis using the fewest resources while maximizing patient experiences, managing and communicating uncertainty to patients, and tolerating watchful waiting when unfocused treatment may be harmful.110 As measurement methods evolve, they should address concepts such as uncertainty and clinician calibration (the degree to which clinicians’ confidence in the accuracy of their diagnostic decision making aligns with their actual accuracy111). Understanding and measuring these concepts in the diagnostic safety equation is essential to optimize the balance between reducing overuse and addressing underuse of diagnostic tests and other resources.112
