Challenges
This research project highlighted several challenges to obtaining consumer-generated reports about safety concerns.1 Safety-related events do not surface in a predictable way, and their causation is complex. This makes standardized surveys and other data collection approaches difficult to design and implement. Moreover, the reporting process must be acceptable to the person making a report. Serious problems may arise during transitions among a wide variety of delivery organizations, creating uncertainty about how best to notify relevant professionals and organizations about patient and caregiver reports. New information technologies are evolving rapidly, so patients and caregivers may not know which channels to use for reporting. The optimal timing for soliciting reports may differ, depending on patients’ health status, their experience, and their expectations. Current systems for classification and coding of events may be useful to clinicians and patient safety managers, but they may not be easily adaptable to the unique perspective that patients and caregivers bring to health care safety improvement. A system for obtaining patient and caregiver reports of health safety concerns must respect current legal and regulatory requirements regarding confidentiality and data protection that come into play when soliciting, storing, and transmitting safety-related information.
Communicating in Patient-Friendly Language
Collecting reports in a language that is both patient-friendly and possible to analyze and categorize in a way that is useful to health care providers to improve patient safety is inherently challenging. The reporting system must communicate to patients in a language they understand. We had hoped to use the AHRQ Common Formats language in the hotline, but we learned in focus groups with patients and family members that the language health care professionals have come to accept as the standard—including such terms as “adverse events,” “medical error,” and “harm”—is potentially confusing or ambiguous to patients. Drawing fine distinctions between errors and harm, making attributions of causality, and assessing impact are all somewhat subtle and nuanced. Furthermore, if the reporting system is to be used to inform health delivery organizations, it must also enable complaints to be categorized and communicated in terms that are useful to risk managers and patient safety officers in retrospective case analyses and for fashioning interventions.
The research team sought patient-friendly language to communicate complex and nuanced information about the purpose of the hotline, how it works, and how patient information would be protected. This challenge was compounded by the requirement to use specific informed-consent language and to include the OMB Paperwork Reduction Act Statement—which raised the overall reading level of the Web content we produced. The informed-consent issue may be an unavoidable challenge for any patient reporting system, but hotlines that are not sponsored by a Federal agency will avoid the additional burden of OMB requirements.
We attempted to use an 8th grade (or lower) reading level in all materials aimed at potential respondents: the patient event reporting form, the Web site pages, and the outreach materials. Ultimately, we achieved a 7th grade, 9th month reading level in the reporting form (without the consent language, it would have been a 7th grade reading level). However, the home page had a 9th grade reading level, and the outreach materials had a 12th grade reading level. One of the health care delivery organizations in the pilot community indicated that it attempts to achieve a 6th grade reading level in all patient-use materials. Doing so with the hotline materials could improve access. A reporting system that is not intended for use in the context of research—such as a quality improvement effort or a public monitoring effort—would be able to achieve a lower-grade reading level.
A related challenge was that of communicating to patients and caregivers the options for sharing information without confusing them or losing their attention. Patients and proxies filling out the form may report anonymously (neither the research team nor the partner organization has their names or contact information) or confidentially (the research team does not share the report with the partner organization); they may report de-identified data (the research team has identifying information but shares a de-identified version of the report with the partner organization); or they may report identified data (the research team and the partner organization both have the report with identified information). The PFACs worked with the research team to develop language that was simple, straightforward, and accurate, such as “You can choose to send the report anonymously or with your name and contact information.” However, having too many options may have led some patients or caregivers to give up before making a report.
Creating Awareness Among Patients and Caregivers (Outreach)
A reporting system is effective only if patients and caregivers know that it exists at the time they have something important to report. Making patients and caregivers aware of the hotline in the pilot project was difficult. The research team, the pilot organizations, and the TEP agreed that a community-wide outreach strategy might not be an optimal model for eliciting patient safety reports, since its impact is diluted across a broad audience. When the hotline prototype launched, partner organizations distributed posters, 4-by-9-inch cards, and business cards throughout their hospitals, outpatient practices, pharmacies, and other facilities. Information about the hotline was also provided in admission packets. Nevertheless, most members of the PFACs indicated that they had not been aware of the hotline. Since PFAC members tend to spend more time than the average patient in health care delivery organizations and are more familiar with activities occurring within the system, the research team concluded that overall awareness of the hotline was low. Information about it may have low salience to most patients, and the promotional information may not have been as clear or understandable as the team intended.
In retrospect, we realize we should have gone to the PFACs earlier in the process to engage them in order to reach out to their communities through local churches or schools or community activities. We could have better utilized the PFACs.
When we did reach out, the PFACs emphasized the difficulty of getting the attention of patients and caregivers during a hospital stay. TEP members emphasized the importance of timing in presenting information to patients. Information may be presented upon admission, during a hospital stay, at discharge, a few days after discharge, and weeks or months after discharge. Each option has benefits and drawbacks. Upon admission, patients and their family members are focused on a number of pressing issues, such as the medical problem that brought them to the hospital, insurance and payment issues, and the retrieval of relevant medical information. An introduction to the hotline at that stage is unlikely to have much effect. Real-time capture of safety concerns during the hospital stay presents an opportunity to fix such concerns before they have a negative effect on the patient. However, a system that allows patients to report in real time would need to focus on the information needed to resolve the issue and would minimize or omit questions that could inform later analyses, such as RCAs. At discharge and a few days after discharge, patients and caregivers are typically no longer concerned about a pressing medical issue and may be more inclined to read information sent home with them. However, they may also be more interested in moving on and reluctant to revisit negative experiences. In sum, there may not be a single optimal time to approach patients with information about patient safety reporting, which is why our outreach strategy emphasized multiple modes and methods.
During the second phase of the project, in response to feedback from the PFACs and TEP members, the research team expanded the avenues through which information about the hotline was disseminated. At one partner organization, information was added to discharge packets and even to post-discharge phone-call scripts for three patient cohorts: congestive heart failure, stroke, and outpatient surgery. However, even after the outreach methods were enhanced, the volume of reports did not increase meaningfully.
Given the relatively high volume of visitors to the Web site (as evidenced by the analysis of Web traffic), there are several potential explanations for the low volume of reports. Visitors to the home page may—correctly or incorrectly—determine that the hotline does not meet their needs. Visitors may have the false impression that the hotline does not offer a way to share a de-identified report with the relevant health organization(s). The home page of the hotline may not be providing sufficiently clear, user-friendly information on the purpose of the hotline and how to complete a report. Finally, the number of visitors to the home page may not provide an accurate count of the number of consumers who were considering making a report because the count also includes visitors who are simply curious about the hotline, including researchers, clinicians, advocates, and lawyers.
Lessons Learned
Given the many challenges associated with the hotline, we learned a number of useful lessons:
- Patient safety reporting is both desirable and feasible. Prior research and the present study demonstrate that patient safety reporting is desirable because it elicits information that may lead to action and improvement within health care organizations. It is potentially empowering for patients and families in that it acknowledges their experiences and perspectives and uses the information they provide to improve care and ensure accountability. Methodologically robust and rigorous tools for eliciting, reviewing, processing, and presenting patient safety reports would benefit health care organizations, public reporting, and patient safety research. The hotline created a mechanism that patients and families used to report meaningful clinical information about perceived errors and physical and emotional injuries. We assessed the feasibility of creating an approach and a platform that could manage a limited volume of reports; additional research may be needed to determine whether this same approach could support a high volume of reports.
- The hotline yielded information that was previously unavailable. The hotline provided information that sophisticated health care organizations would otherwise not have had, despite the existence of other mechanisms for eliciting patient safety reports (e.g., patient experience surveys and complaint departments) and information about patient safety concerns from staff (e.g., adverse event reporting systems). The hotline reports were more detailed and richer than information collected through other reporting mechanisms and included previously unreported events. The reports were credible and led to actions—they were just few in number.
- The hotline was readily incorporated into existing patient safety systems and was not disruptive to health system operations. Two feared scenarios never materialized. First, hospital administrators had worried that they would not have sufficient bandwidth to respond appropriately to all reported patient concerns. Instead, they ultimately tried to elicit more reports from patients. The hotline was not only manageable for participating health systems but was readily incorporated into their patient-relations and patient safety systems. However, having a third party (RAND and ECRI) process the reports before sharing them with the organizations—as was done in the hotline pilot—may not be an ideal arrangement, since the organizations goal is to be in contact with the patients as quickly as possible after an event occurs. Second, some leaders were concerned at the outset that the hotline would be disruptive; this was not the case, and its implementation actually reinforced organizational commitment to safety, quality, patient engagement, and transparency.
- Legal and regulatory obstacles to soliciting, storing, analyzing, and sharing event reports are manageable through the use of PSOs. Together with AHRQ, we investigated the nature of various protections for patient-reported information early in the project, concluding that such information was covered by AHRQ research protections. Data from individual health care organization hotlines can be protected in a PSO/PSES environment. These obstacles are likely to be even less problematic if hotlines are adopted by individual health care systems for their own use.
- Responses to open-ended questions were considerably more useful than the responses to structured questions that followed. The physicians who classified the patient event reports according to event type, harm, duration of harm, preventability, and contributing factors found that most of the value of patient reports lies in the narrative. This finding is consistent with findings of previous studies and with safety reporting by professionals in health care and aviation.
- Although the narratives were most helpful, collecting structured report elements from patients offers potential benefits. To the extent that we can standardize questions and validate that reporters interpret the questions and response elements in a consistent way, the use of structured questions would facilitate data aggregation, preparation of summaries, and use of statistical analytic tools. However, this is an area that requires additional research.
- The option of proxy reporting, in which family members or other caregivers report on safety concerns, was an important component of the hotline project. Proxies submitted about half of the hotline reports we received.
- The platform we developed has a number of components that could be used, depending on the specific goals of the end user. Which components are employed and how they are adapted would depend on the user’s purposes.
Although the hotline had many positive features and benefits, the potential for broader adoption or deployment of consumer reporting systems remains uncertain. While technical, regulatory, and organizational issues appear to be tractable, the challenge of creating the opportunity to report at a time and in a way that motivates patients or caregivers to do so has not yet been satisfactorily resolved. The research team, our partners, and the TEP considered several future scenarios involving patient safety reporting, including:
- Incorporating patient safety screening and/or survey questions into routine patient experience surveys (CAHPS, Press Ganey, and others) that are already used in inpatient and ambulatory settings.
- Incorporating safety screening into patient portal-based surveys that are keyed to emergency department visits, hospital discharge, or office encounters.
- Enrolling and training patients with chronic conditions (e.g., chronic obstructive pulmonary disorder, cancer, congestive heart failure, diabetes, cystic fibrosis) as professional reporters, a kind of “Nielsen rating” program for health care.
- Identifying family members of patients with high-risk conditions (e.g., neonatal intensive care patients, patients with pediatric cancers) who may be motivated to make real-time reports.
- Placing tablets or kiosks in ambulatory care areas to facilitate reporting, or creating a mobile app that can be used in real time in any setting.
