Patient Experience as a Source for Understanding the Origins, Impact, and Remediation of Diagnostic Errors

To strengthen our future capacity to learn from patients’ diagnostic experiences, we identify four challenging questions that have been posed in the existing literature but not yet adequately addressed. For each question, we offer some concrete, feasible responses to these challenges.

Who Should Report Their Diagnostic Experiences?

The Challenge: When eliciting accounts from patients and families, one cannot simply ask respondents if they have experienced a diagnostic error. Rather, the data collection method or instrument must define what is in scope to report. Although many Americans describe having heard about “medical errors” in the media,¹⁵ only about half report that they have any sense of what the term means, and a quarter have never heard the term at all.¹⁶

Conventional research practice has been to define “medical error” or “diagnostic error” within the survey. For example, in two large national surveys of Americans’ experiences with patient safety, researchers at the Institute for Healthcare Improvement (IHI) used this definition:

Sometimes when people receive medical care, mistakes are made. These mistakes sometimes result in no harm, while other times they may result in additional or prolonged treatment, emotional distress, disability, or death. These types of mistakes are called medical errors.¹⁶

This definition is clearly worded and coherent. But clarity of language does not necessarily help respondents determine whether an adverse event experienced in a clinical setting was caused by a clinician’s mistake or error. Respondents with limited education or from cultures that discourage challenging professional authority may be particularly reluctant to draw this conclusion.^3,17 These barriers can induce or exacerbate disparities in “error” reporting.

Potential Response: Given the challenges with language, invitations for the public to report on diagnostic experiences should generally avoid asking respondents to determine whether their experiences fit some predefined criteria for medical or diagnostic errors. If a survey instead asks respondents about “feeling unsafe” during their experiences as patients, a much larger portion of the public will respond. In hospital settings, this shift in terminology from mistakes/errors to feeling safe roughly doubled the number of patient-reported safety events.¹⁸

Although it seems clear that elicitation methods should avoid labels such as “medical error” or “diagnostic error” when inviting reports about patient experiences, the preferred alternative is less apparent. Asking about “feeling unsafe” extrapolates imperfectly to diagnostic experiences and may be unduly influenced by respondents’ prevailing level of trust in the healthcare system. If patients from certain backgrounds or with particular prior experiences are more likely to mistrust healthcare providers, they may never “feel safe,” regardless of their recent experiences.

One promising alternative would be to ask patients about “mistakes and problems” in obtaining a diagnosis, although this option has yet to be tested in any studies in the peer-reviewed literature. Framing experiences in terms of “mistakes” is more approachable than framing as “diagnostic errors,” since an error is seen by patients as implying culpability for the problem. Expanding the scope to also include “problems” encourages reporting of adverse events where patients and families do not necessarily blame clinicians for negative outcomes.⁶ Whatever the specific wording, a sensible goal would be to broaden the scope of experiences being reported, in order to collect actionable feedback about as many diagnostic mishaps or concerns as possible.

Which Aspects of Diagnosis-Related Experiences Should Be Elicited and Reported?

The Challenge: Most studies eliciting patient experience with medical errors focus on the adverse event itself: the nature of the perceived medical mishap, the factors associated with its emergence, and the immediate harms it caused.¹⁹ To illustrate, the largest survey to collect narrative accounts from hospitalized patients about their safety concerns did so through a sequence of open-ended questions¹⁸:

1. Please tell us what happened with your concern or experience in as much detail as you can.
2. Why do you feel this was a safety concern for you?
3. What do you think could be done to stop this from happening again, to you or other patients in the future?

This type of wording is the norm for collecting patient narratives related to safety issues. The focus of questions is on the immediate antecedents and outcomes of the adverse event. By narrowing the scope of inquiry in this manner, researchers hope to have patients and families report on the aspects of their experience that are closest to the adverse event in question.^10,18,20,21

But this approach has notable shortcomings. First, it focuses only on the actions, attitudes, and circumstances associated with preventable events. Given the uncertainties inherent in diagnosis, a portion of adverse diagnostic events that are not always preventable would be missed. Those that are identified in research as not being preventable have been shown to affect patients’ well-being as severely as preventable errors do.²²

A second limitation of focusing solely on the adverse event is that the ways patients and clinicians interact in the aftermath of that event go unreported. The literature on medical errors documents that the long-term harms associated with adverse events are often worsened by clinicians’ actions after the event. Some cases involve actions they fail to take, while others involve actions clinicians or healthcare systems do take that make matters worse for patients and their families.^23,24

Conversely, the negative consequences that follow medical errors and adverse events can be partly mitigated by effective communication in the aftermath of the incident.^25,26 Although these remedial benefits from communication have been amply documented for treatment errors, we know less about their benefits after diagnostic mishaps. More effective learning from patient and family accounts could help identify whether and when mitigation efforts were more or less effective for diagnostic-related experiences.^27,28

Potential Response: Because past studies of perceived diagnostic mishaps have not focused on interactions between patients and clinicians after the adverse event, the peer-reviewed literature has no evidence about their prevalence or impact on patient well-being. Nor do we have any evidence on how best to elicit experiences in the aftermath of a diagnostic mishap.

However, we can build on the small body of prior research that asked respondents about their experiences in the aftermath of other medical mishaps.^23,24,25,26 Although these were largely focused on adverse treatment events, some also collected data related to perceived diagnostic problems.

In a prior study of experiences with medical errors in Massachusetts, we documented the lasting sequelae of medical errors by asking explicitly about the clinician actions that might have “improved the situation following the medical error.” We also asked about clinician actions following the perceived medical error that “made things worse.” In studies published on these findings, we did not separate out the impact of perceived diagnostic errors.²⁵

If we now parse out cases involving perceived diagnostic failures, it becomes clear that diagnostic mishaps also have persisting consequences. Considering only respondents for whom the diagnostic problem happened 3 or more years in the past, 28 percent still report having felt abandoned by their clinicians after the adverse event. In addition, 22 percent report feeling anxious when they access medical care. Finally, 78 percent express a continued loss of trust in clinicians to protect against future diagnostic errors.²⁵ Illustrative accounts of these experiences, in patients’ own words, are shown in Exhibit 1 (examples are verbatim and have not been edited).

Exhibit 1. Illustrations of persisting sequelae of diagnostic mishaps reported in elicited patient narratives

Virtually every respondent who reported a persisting harm identified at least one action they wished their clinicians had taken in the aftermath of the diagnostic problem. In addition, nearly half (46%) of those who reported persisting harms noted at least one clinician action after the diagnostic mishap that they felt made matters worse, including, most often, clinicians refusing to acknowledge that the diagnostic process had gone awry. Patients and families saw this stance as dismissing or denigrating their experiences.

Thus, it seems imperative to elicit patient experiences in ways that generate learning about the extended aftermath of adverse diagnostic events. For completeness, elicitation questions should inquire specifically about clinicians’ postevent actions, practices, and policies that:

• Improved patients’ well-being after a diagnostic shortfall.
• Exacerbated harms associated with the adverse event.
• Might have reduced the harms if those actions had been made available to patients and families who had experienced the diagnostic shortfalls.

How Should Timeframes Be Handled for Eliciting Diagnostic Experiences?

The Challenge: To assess the longer term aftermath of perceived diagnostic mishaps, one must elicit patient experiences about events that occurred multiple years in the past. This need poses certain additional challenges related to potential reporting biases (e.g., recall, attrition) and the costs required to reduce these biases.

Research on treatment errors suggests that aftereffects can persist up to 5 to 10 years.^25,26 How best can we assess such long-persisting sequalae? Two general approaches are possible.ⁱ The first identifies a panel of respondents who report experiencing a diagnostic mishap in any one year, then follows them over time, repeatedly collecting information about their current well-being.

This approach is challenging and costly to implement. Substantial attrition is inevitable among panel participants over 5 to 10 years. Remaining participants may represent a biased subset of the original cohort, distorting the prevalence and distribution of long-term effects and how clinician practices and organizational factors shape them. These challenges are sufficiently daunting that, to our knowledge, no one has conducted longitudinal studies of diagnostic or medical mishaps.

The second approach involves collecting patient experiences at a single point in time, asking about diagnostic events that occurred several years in the past. Population-based surveys of medical errors typically have had “look-back” periods of up to 5 years.^16,25,29 With this approach, about 23 percent of respondents report having experienced a medical error in their own care or that of a family member or friend over the past 5 years.¹⁶ Stratifying elicited experiences based on how many years earlier the adverse event occurred makes it possible to construct a profile of how the consequences of the events change over time.²⁵

Cross-sectional surveys that ask respondents to look back in this manner also have limitations. Chief among these is recall bias: the farther back into the past respondents are asked to recall events, even memorable ones such as medical mishaps, the more likely they are to forget some experiences.

For example, on the 2017 survey IHI fielded, 9 percent reported that they or someone they know experienced a medical error in the past year. But only 14 percent more reported such an event from 2 to 5 years before the survey (less than half of what one would expect if a 9% annual rate were replicated for each of the 4 previous years). Since mishaps with minor consequences are more likely forgotten than those with large impact, recall bias threatens to also distort understanding of the true prevalence of various effects and aftereffects.

Potential Response: Although longitudinal studies have the greatest potential for tracking the extent of and influences on aftereffects, their complexity and cost make them largely infeasible for monitoring populationwide effects of diagnostic problems.ⁱⁱ This limitation leaves cross-sectional surveys that ask respondents to look into their past to identify diagnostic events as the most realistic approach for monitoring diagnostic experiences at large scale. To deploy cross-sectional surveys in this way, it is essential to address the problems created by recall bias.

Fortunately, a plausible approach exists for doing so.²⁵ Questions about perceived diagnostic shortfalls in the past can ask patients about the impact of diagnostic problems both in the immediate aftermath of the event and those currently experienced. One would predict from prior research on recall bias that events having more modest short-term impact are more readily forgotten in the long term. If this assumption is valid for diagnostic mishaps, the immediate impact reported for diagnostic problems experienced multiple years in the past would reflect a larger percentage of serious harms than were reported for events experienced within the past year.

One can assess the magnitude of recall bias in this manner. Moreover, these estimates can then be used to statistically adjust for the impact of recall bias on the prevalence of reported experiences. Using reported prevalence of harms from diagnostic events within the past year to “reweight” the frequency of minor and major diagnostic events from the more distant past makes it possible to offset some of the biases caused by uneven recall of more distant diagnostic mishaps. Although this method does not eliminate concerns about recall bias for retrospective questions, it can at least mitigate some concerns.

How Do We Encourage the Most Complete Reporting of Diagnostic Experiences?

The Challenge: Most studies of narrative reporting about safety experiences have drawn on accounts volunteered by patients and families.^iii,3,23,26 That leaves open the questions of who might be reporting and how completely their reports convey all aspects of their experiences.

Patients and families may unevenly report experiences for many reasons. They may unevenly convey positive versus negative experiences, because they anticipate that one or the other will be more likely to induce a constructive response from clinicians.^30,31 Some patients may fail to report their experiences at all, doubting that doing so would yield meaningful change. If patient narratives are less frequently or extensively reported by respondents who have limited trust in medical care, uneven reporting may exacerbate disparities in diagnostic shortfalls.

One option is to add questions about diagnostic safety to existing patient experience surveys. However, this approach has several challenges. First, when patients experience adverse medical events, they frequently switch to alternative venues for their subsequent medical care.³² Studies suggest that up to half of all patients and families switch providers in the aftermath of an adverse medical event.^iv,25 Therefore, clinician-based surveys of current patients will understate the prevalence of diagnostic problems as well as their impact on patients and families, since many of those experiencing problems will have already switched to another clinician.

If switching is most pronounced when diagnostic problems have created the most serious harms for patients and families, the most impactful adverse events will also be the most undercounted by surveys fielded by the delivery systems within which the problem emerged.

The second challenge to collecting accounts about diagnostic mishaps through existing patient experience surveys relates to survey format and timing. Surveys that are distributed shortly after a discharge or a visit might  be premature for assessing diagnostic issues, which often take some time to resolve and sometimes even longer for patients and families to sensibly interpret.³³

Surveys asking about experiences over a set timeframe, such as 6 to 12 months, allow more reflective, retrospective questions that aid in assessing the aftereffects of a perceived diagnostic mishap. However, even a 12-month look-back period is insufficient to track some of the more persisting sequelae that follow these mishaps.

The challenge of collecting detailed, representative narrative data is thus twofold. First, it is essential to identify a source for sampling experiences that will ensure a representative group of patients and families is appropriately motivated to participate. Second, elicitation methods must be structured to ensure the accounts are collected in ways that allow patients to reflect on their diagnostic experiences. For example, patients need enough time to assess when diagnoses were appropriate, timely, and effectively communicated.

Potential Response: To address the first of these challenges, it is essential to collect information about diagnostic experiences through multiple channels. To collect information that is most immediately actionable, incorporating questions about diagnosis into encounter-based patient experience surveys or those that rely on a relatively short look-back period holds promise. However, these forms of elicitation will miss the experiences and distributional patterns of patients who have switched clinicians in response to their diagnostic mishap. Such patients are too focused on short-term experiences to provide much understanding of the more persisting implications of diagnostic shortfalls or how those perceived mishaps are addressed.

Several principles for robust narrative elicitation emerged from various studies. First, many otherwise willing respondents find it difficult to convey a coherent account of their healthcare experiences. The questions they are asked need to be carefully sequenced to help patients reflect on their experiences, recount events in the order in which they occurred, and consider both positive and negative aspects of their experiences.¹³

Second, more complete narrative accounts were elicited when the questions about patient experience incorporated a form of “scaffolding,” question wording that helps respondents consider multiple ways to respond to the query.¹⁴ For example, rather than simply asking patients about their positive or negative experiences with clinicians, scaffolding questions asked about these encounters and encouraged respondents to “explain what happened, how it happened, and how it felt to you.”

The impact of this scaffolding approach on question design was most pronounced for respondents with less experience offering their accounts about healthcare. This group includes historically marginalized populations: those who are economically disadvantaged, those with stigmatized health conditions, or those who have previously experienced perceived discrimination when interacting with clinicians and health systems.14 The impact of question design was particularly evident for the extent to which narratives conveyed “actionable” information for improving care in the future.³⁴

Elicitation questions designed in accord with these principles hold the promise of encouraging the most coherent and complete narrative responses. But that does not guarantee that these responses will reliably convey all aspects of patients’ healthcare encounters. To test this reliability, the content of elicited narratives needs to be compared with the more complete accounts about patient experiences that can be obtained from intensive interviews conducted by interviewers trained to actively encourage the most complete accounts.¹³

Synthesizing Insights About Rigor in Eliciting Patient Experiences With Diagnosis

We have identified above four key challenges to eliciting narratives about diagnostic experiences, along with ways to constructively address those challenges:

• Potential respondents should be encouraged to report diagnostic experiences that felt problematic or threatening to their safety, rather than those narrowly defined as medical error.
• Open-ended questions should be designed to provide information about both the emergence of the problem and its consequences in the aftermath of the event.
• Questions should be worded in ways that provide scaffolding that encourages more complete and balanced responses, especially from those with less experience recounting their medical experiences. Those sequences of questions need to be tested to assess which aspects of diagnostic experience can be reliably elicited and which cannot be.
• The sequences of elicitation questions, once carefully designed and tested, should be incorporated into surveys appropriate to the potential uses for that feedback. To identify and respond to diagnostic shortfalls that are most immediately actionable, elicitation questions might be most appropriately incorporated into existing patient experience surveys based on one-off encounters or with short look-back periods. However, to provide a more complete assessment of the prevalence, distribution by population subgroups, and impact of diagnostic shortfalls, it would be essential to include elicitation questions as part of population-based surveys, not just those fielded by specific organizations. These findings are summarized in Exhibit 2.

Exhibit 2. Recommendations regarding elicitation of patient experience

^i. Presenting these as two options is a simplification. Many hybrids involve some combination of tracking experiences and asking retrospective questions. The simplified dichotomy illustrates the issues and is not intended to define all possible survey design responses.

^ii. Longitudinal studies may still be feasible for addressing more limited objectives, if incorporated into existing longitudinal surveys or fielded for smaller, stable populations of respondents.

^iii. The exception is collected narratives during the course of a stay in the hospital.^8,18 These demonstrate that narratives of patient experience can be collected from a broader cross-section of the public but share the shortcomings discussed in the text. That is, they lack a way to validate whether the information being reported in the narratives reliably replicates the events respondents actually experienced.

^iv. To provide a sense of how much switching occurs after a diagnostic mishap, we can again separate out the diagnostic problems from among the medical errors reported in our earlier study of Massachusetts residents.²⁵ Fifty-nine percent of respondents who reported diagnostic problems switched doctors after the event; a comparable percentage switched hospitals if they had a diagnostic problem at that facility.