Cognitive Load Theory and Its Impact on Diagnostic Accuracy

To best understand cognitive load’s impact on diagnostic reasoning, researchers and system improvement experts must be able to measure it. When CLT was first conceptualized, there were few ways of directly measuring cognitive load.¹⁵ Since then, numerous measurement tools have been developed with the goal of more directly measuring the effects of various interventions on cognitive load. Still, accurate measurement remains a persistent challenge due to the inherent difficulty of measuring an entirely internal cognitive environment.⁴⁰

Measures that do exist can generally be broken into subjective, objective, and electronic measurement tools. Below is a summary of each of these tools; applications in healthcare studies will also be discussed, although research using these tools to directly study diagnostic accuracy is lacking. To better design studies that can causally link unsafe levels of cognitive load with diagnostic error, an understanding of the available measurement tools is needed.

Subjective Measurement Tools

Subjective measurement tools are generally represented by retrospective surveys and questionnaires that aim to measure the respondent’s perceived workload or mental effort. The NASA-Task Load Index (TLX), Subjective Workload Assessment Technique (SWAT), and Paas Cognitive Load Scale were some of the first subjective measures created in the late 1980s and early 1990s and are still widely used today.

The NASA-TLX⁴¹ is used across multiple fields, including aviation,⁴² healthcare,⁴³ human factors engineering,⁴⁴ and industrial ergonomics.⁴⁵ Importantly, the NASA-TLX measures self-reported mental demand and assesses physical demand, temporal demand (i.e., time pressure), performance, effort, and frustration to provide a holistic view of overall workload.

The SWAT is a multidimensional tool created to capture the impact of time load, psychological stress load, and mental effort load to create an overall score for perceived mental workload.⁴⁶ The Paas Scale is a 9-point Likert-based tool that can be used for self-evaluation of the cognitive effort required for a task.⁴⁷

The NASA-TLX, SWAT, and Paas Cognitive Load Scale can provide indicators of overall self-assessed cognitive load⁴⁸ but are fundamentally interruptive to administer and, by nature, retrospective, which limits their applicability in pragmatic clinical environments.

Surprisingly little research has been done using subjective measurement tools to study if perception of rising cognitive load is associated with increased risk of diagnostic errors. As discussed above, cognitive overload is associated with decreased cognitive flexibility and more simplistic reasoning,³⁹ which may mean that high levels of cognitive load are an important variable associated with diagnostic errors; causal research into this area is still needed.

Subjective measurement tools have been used to research workload and its associated impact on patient safety in other domains, however. For instance, a recent study found that increased nursing workload in the neonatal intensive care unit was associated with missed nursing care. Interestingly, when the missed care variables were modeled independently, only 7 of the 12 missed care outcomes showed that increased nursing-to-infant ratios were associated with missed nursing care. However, higher NASA-TLX scores were associated with an increased risk of missed care in all 12 missed care outcomes.⁴⁹ Higher mental workload can also lead to worse surgical laparoscopic performance⁵⁰ and increasing educational workload is associated with reduced vigilance during anesthesia induction.⁵¹

Objective Measurement Tools

More objective measures of cognitive load have also been developed; like their subjective counterparts, they cannot accurately measure cognitive load subcomponents but can provide an indicator of overall cognitive load at the time of the task, rather than retrospectively. They can broadly be divided into nonphysiologic and physiological measurements.

Common nonphysiologic measurement tools that can be used to study cognitive load are:

• Dual-task methodology (asking a participant to perform two tasks concurrently, with the goal of assessing whether performance drops across either task).⁵²
• Trail Making Test (accurate timed connection of nonsequential dots).⁵³
• Digit span recall (accurate repetition of a sequence of numbers).⁵⁴
• Stroop task (accurate recitation of mismatched ink color and word).⁵⁵
• Wisconsin Card Sorting Test (accurate card sorting based on stimulus cards and rule-based feedback).⁵⁶

These tools have commonly been used to study cognitive load and working memory, including recall, error rates, sustained vigilance, and processing speed. A significant portion of the literature focuses on the effect of sleep, shift work, distraction, and fatigue on cognitive function. For example, a great deal of literature links multitasking and fatigue with prescribing errors,¹² cognitive overload with poor performance in simulation training environments,⁵⁷ and rapid shift rotation with impaired perceptual and motor abilities.⁵⁸

As with subjective cognitive load measures, surprisingly few high-quality studies have been conducted to objectively study increasing cognitive load and the risk of diagnostic errors, despite a large body of literature linking cognitive overload with other types of medical errors.

More physiologically based measurement tools have also been validated as ways of measuring cognitive load. Eye-tracking studies have shown that pupil diameter and the rate and magnitude of microsaccades change in reliable ways depending on task difficulty and cognitive load.⁵⁹ One study found that novice physicians showed higher pupillary responses during difficult clinical questions compared with more expert physicians.⁶⁰

Heart rate and heart rate variability have also been correlated with high cognitive load. Increasing rates were found to correspond with increasing measures of intrinsic cognitive load along with worsened performance during clinical reasoning tasks.⁶¹

Electronic Measurement Tools

EHRs have introduced a significant change in clinician workflow, with up to 50 percent of a clinician’s day now spent in front of a computer.⁶² Clinicians also report increased interruptions and alert fatigue from poorly designed decision support tools.⁶³ Theoretically, EHRs can streamline workflows, reduce cognitive load via clinical decision support algorithms, and improve patient safety. But they can also lead to inefficiencies, cognitive overload, and safety events.⁶⁴

Understanding the impact of EHRs on cognitive load and diagnostic accuracy will be critical to advancing patient safety. Several EHR cognitive load measurement tools include EHR audit log data and EHR usability scales.

Audit log data provide an unobtrusive way to measure clinician workload and workflow. These logs contain time-stamped data about how a clinician is interacting with the EHR and can capture what tasks are performed and the time spent on them. Examples include chart reviewing, use of clinical decision support tools, methods of documentation, and choice of orders.⁶⁵

A validated submeasure that uses EHR audit log data is called the “Wrong-Patient Retract-And-Reorder” (Wrong-Patient RAR) tool. It quantifies how many orders are placed for a patient, retracted within 10 minutes, and then reordered by the same clinician for a different patient. Although this type of error would be considered a “near-miss” since it never reaches the patient, research has shown that near-misses can serve as an early indicator of more serious system faults. Similarly, improvement efforts that decrease the RAR rate should also decrease the number of wrong-patient orders that do reach patients.⁶⁶

EHR audit log data can also be coupled with radio-frequency identification, which is a sensor-based technology that enables movement tracking, to create a comprehensive understanding of clinician workflow, including possible insight into cognitive burden. For example, one study found that some emergency physicians read several charts at one time and then visited multiple patient rooms before returning to document the encounters. Audit log data found that these physicians were less efficient and spent more time documenting this type of encounter, possibly due to higher cognitive burden.^65,67,68

Audit log data allow a detailed analysis of clinician workflow and workload. Future research should focus on correlating workflow and workload with cognitive load, potentially using some of the aforementioned subjective and objective tools, followed by assessment of the impact on diagnostic accuracy.

EHR usability can also be measured via cognitive task analysis. Cognitive task analysis assesses performance both pragmatically and in simulations and explores how people think. This type of measurement allows assessments about how electronic interfaces may increase extrinsic load and contribute to cognitive overload.^69,70

Cognitive task analyses have also been used to show that as configured, many EHRs do not help clinicians maintain a “big picture” of the patient, including their current state and what treatments may help.⁷¹ Measuring the impact of technology on clinical workflow is a newer area of research. Given the degree to which technology is now interwoven with clinical care, however, it will be important to understand the impact of this technology on cognitive load, diagnostic error, and patient safety.

Proposed Foundational Research

As discussed above, high-quality studies exploring the causal relationships between cognitive load and diagnostic accuracy are lacking. We propose the following two areas as the most immediate to focus on to create a better foundational framework for understanding cognitive load and diagnostic accuracy:

1. More robust investigation into how high cognitive load impacts clinician diagnostic accuracy in simulated settings. While simulated environments do not perfectly reproduce clinical environments, they allow more precise and replicable manipulation of clinically relevant independent variables intended to increase cognitive load. Such variables may include interruptions and sensory stimuli designed to be distracting, including disruptive electronic messages/pages and alarms.

   Participants could be asked to work through standardized clinical vignettes while being exposed to stimuli designed to increase their cognitive load. Researchers should confirm that these stimuli increase cognitive load via subjective and objective measures (e.g., NASA-TLX surveys and heart rate variability). Once researchers establish that these stimuli reliably increase cognitive load, participants could work through a variety of clinical vignettes while the type and number of diagnostic errors are measured.

2. Pragmatic investigation into how real clinical environments impact clinician cognitive load and diagnostic reasoning. While pragmatic studies can be more difficult to standardize and results can be difficult to generalize, these studies would allow better understanding of how cognitive load impacts clinicians in real patient care settings. As with simulated studies, pragmatic studies could incorporate both subjective and objective cognitive load measures. Clinicians could be asked to wear heart rate monitors that capture heart rate variability and to fill out NASA-TLX surveys at the end of each shift.

   Diagnostic errors are often not identified until after patient harm has occurred (e.g., a missed pneumonia diagnosis may not be recognized on the day that it is missed, but rather when the patient develops complications several days later). Thus, chart reviews could occur after a predetermined amount of time has elapsed to assess for the presence of diagnostic errors.

   Assessments could be done by the treating clinician to allow self-identification of errors, along with a chart review by an independent expert reviewer. The NASA-TLX and heart rate variability scores could then be compared with the number and severity of diagnostic errors.

Once the causal relationships between cognitive load and diagnostic accuracy have been better explored and defined, healthcare systems and researchers can begin to investigate more practical ways to help clinicians optimize their cognitive load to enhance their diagnostic accuracy.