Improved Diagnostic Accuracy Through Probability-Based Diagnosis

Figure 1 depicts the steps in the diagnostic pathway and describes the probabilistic skills and information needed for each step. It shows the centrality of probabilistic understanding to making diagnoses at each of the following steps:

• Step 1: Chief complaint and initial differential diagnosis. The diagnostic pathway begins with a chief complaint from a patient. The clinician creates an initial differential diagnosis based on knowledge of disease incidence in the population of interest (i.e., the population the patient is from). This estimate serves as the initial “pretest probability.”

• Step 2: Adjustment based on history and physical exam. The clinician performs a history and physical examination, each element of which informs adjustment of the probabilities of diseases in the differential diagnosis, ending in a patient-specific differential diagnosis. This step requires knowledge of the impact of each history and physical exam finding on probability (e.g., test accuracy), expressed as sensitivity and specificity or likelihood ratios.

• Step 3: Selection of diagnostic tests. The clinician then decides to perform a particular diagnostic test (or set of tests) to explore likely diagnoses. The chosen tests may relate to the most likely diagnosis or the most concerning diagnosis, depending on the clinical scenario. Deciding to order a test requires understanding of the probability that the patient may benefit or be harmed by getting the test.

  While many clinicians do not frame diagnostic testing in terms of patient benefit and harms, tests, like all other health services, will either help patients or harm them. For example, a test may identify a diagnosis for which treatment improves outcomes or it may expose a patient to toxic substances, inconvenience, or unnecessary care for which harms outweigh benefits. These benefits and harms vary widely in magnitude.

• Step 4: Test interpretation. Once a test is performed, the clinician must interpret the results in the context of the pretest probability to arrive at a posttest probability. This step requires understanding Bayes Theorem, which integrates measures of test accuracy into the pretest probability and requires rejecting the notion that test results are definitive.¹⁵

  While explicit calculations using Bayes Theorem may not be feasible during clinical practice, conceptual understanding of Bayes Theorem informs clinical thinking. For example, a 40-year-old woman with no cardiac risk factors and nonspecific chest pain has an abnormal exercise stress test. She remains unlikely to have coronary artery disease, because her pretest probability was so low. If multiple tests are done, the results of each should be considered when calculating the ultimate posttest probability.

• Step 5: Final diagnosis or further testing. Finally, the clinician must decide when diagnostic closure is achieved to complete the diagnostic phase. This step involves determining whether a diagnosis is established, i.e., considering whether the posttest probability is high enough (or the uncertainty is low enough) to begin management. The disease probability at which that threshold is crossed (the “treatment threshold”) will vary based on characteristics of the disease and its treatment, as well as clinician and patient risk tolerance.¹⁶ If the likelihood of disease is lower than the threshold, further testing may be appropriate.

Correct diagnosis, then, relies on accurate estimates of pretest probability and understanding the influence of positive and negative tests on that probability. However, clinicians generally overestimate the chance a patient has disease under consideration, both before and after testing.¹⁷ This tendency likely leads to misdiagnosis of conditions from false-positive test results and subsequent missed diagnoses that are truly causing symptoms, with potential for patient harm.

Reasons To Refocus Training on Probability

While the need to weigh probabilities is widely accepted as foundational to the diagnostic process and medical students may receive instruction in test accuracy, clinically integrated training in probabilistic diagnostic reasoning is lacking.¹⁸ Learning about probability has mostly been limited to memorizing definitions of sensitivity and specificity and calculating them from studies of test accuracy using 2X2 tables.

Errors in estimating probability of disease may arise from this approach¹⁹ as mathematical calculations are difficult to apply to clinical medicine.²⁰ Indeed, most teaching about test interpretation and probability happens in the preclinical years of medical school, suggesting that it is separate from the approach to testing used in daily clinical practice.

Further, in clinical practice, most medical decisions are made rapidly and intuitively,²¹ so it is critical for clinicians to make accurate “gestalt” estimates of pretest probabilities of common disorders and intuitively adjust pretest probabilities based on test results.²² These estimates must be updated with each subsequent test. At the same time, for less common presenting complaints or syndromes, for which clinicians have little intuition, they must use quantitative estimates to make decisions.

Currently, little evidence is available to inform approaches to teaching diagnostic reasoning and the most common discussions of diagnosis for trainees occur in the context of generating differential diagnoses.²³ These discussions do not emphasize probabilistic understanding. In fact, they may undermine intuitive understanding of prevalence and probabilistic reasoning by rewarding learners for suggesting rare diseases with extremely small likelihood.

Methods To Teach and Inform Probability for Diagnostic Decision Making

To appropriately incorporate probabilistic thinking into the diagnostic process, clinicians need to both have ready access to the variety of critical quantitative data that is currently difficult to obtain and intuitively understand probability. Figure 1 shows the probabilistic information needed at each step in the diagnostic process and current needs to fill the gap.

• At Step 1: Developing initial differential diagnosis: Provide easily accessible incidence data for common diseases. When estimating an initial differential diagnosis based on the chief complaint, clinicians need ready access to incidence data in the local community. This information may be challenging to obtain but it is possible. For example, information about local rates of COVID-19 can be easily accessed online (e.g., through the Centers for Disease Control and Prevention’s COVID Data Tracker) and are useful to facilitate test interpretation.

• At Step 2: Adjustment based on history and physical: Provide sensitivity and specificity of history and physical examination data. When adjusting the initial differential based on elements of the history and physical, clinicians need evidence of the sensitivity and specificity of clinical features and physical examination findings, which are largely unavailable. In addition, they need to understand how to adjust probability based on test results, whether those tests are physical examination maneuvers or blood or imaging tests. Such information is difficult to find but exists most prominently in JAMA’s Rational Clinical Examination series at https://jamanetwork.com/collections/6257/the-rational-clinical-examination.

• At Step 3: Selection of a diagnostic test: Provide information on probability of potential benefits and harms of common tests in absolute terms that is readily available at the point of care. While few clinicians currently conceptualize tests through a lens of benefits and harms, the approach to screening tests is an illustrative exception. For example, using currently available decision aids, we can frame decisions around PSA tests as a balance between potential benefits (e.g., reduction in prostate cancer deaths or advanced disease) and harms (e.g., urinary incontinence and erectile dysfunction). Similar logic could be applied to testing more generally with better access to information.

• At Step 4: Test interpretation: Teach about test accuracy using natural frequency interpretation via games or other novel methods. Perhaps most importantly, clinicians need better intuitive understanding of probability. Teaching probability can be most effective when students are exposed to natural frequency figures.²⁴

  Instead of tables with probability calculations, approaches can be grounded in patient populations. For example, students may be asked to consider 100 identical patients, then divide them by the percentage likely to have (representing pretest probability) or not have disease. The number of positive tests that would occur in patients with disease and those without disease could be estimated based on incidence and sensitivity and specificity, respectively.²⁵ These estimates can be depicted using graphic images showing grids or icons representing risks out of 100 or 1,000 people; such illustrations have been found to work better than other depictions in those with less training.²⁶

  To our knowledge, these techniques have not been widely studied or adopted in medical training. In addition, repeated practice with clear, actionable feedback is a classic and effective learning approach that can help clinicians develop an intuitive sense of probability.^27,28

• At Step 5: Final diagnosis or further testing or consideration of other diagnoses: Acknowledge uncertainty and teach methods for determining thresholds. Diagnostic error in clinical medicine exists in part because of the inherent uncertainty that stems from the great diversity in patient symptoms and findings and the lack of clarity around many diagnoses. Clinicians confronting this uncertainty seldom receive feedback about the assumptions that underpin their diagnoses, which can reinforce faulty reasoning.

  Educators themselves may need instruction in managing uncertainty.²⁹ Probability thresholds for testing and treating may vary by individual and geographic region, but appropriate ranges can be estimated based on survey studies.³⁰ Probabilistic treatment thresholds should be included in all clinical discussions of plans for diagnostic testing. This information can reinforce the importance of probability and provide feedback that can fine-tune learners’ sense of appropriate thresholds for testing or treatment in different contexts.

Novel delivery of classic learning approaches can serve as an effective method for teaching these skills. For example, games with a primary educational goal, known as “serious games,” use repetitive, rapid decision making with immediate feedback to train skills.³¹ They have been widely used to improve skill in areas such as chess and gambling. More recently, these games have moved to medicine, where successful applications have included patient care simulations in emergency surgical settings. Such games can be more efficient than standard problem-based learning and may be superior for training intuitive skill (vs. improving knowledge).³²

Most games have focused on discrete lessons related to individual cases, with few trying to develop a general skill. More recently, games have targeted heuristics to change thinking processes inherent in clinical medicine, suggesting broad future application.^33,34 One such game resulted in durable improvement in appropriate triage of trauma patients in an emergency department.³³ Serious games have potential to facilitate achievement of diagnostic excellence in medicine by motivating repetition and feedback and could be used both during training and by practicing clinicians, ideally for continuing education credit.