Demonstration Grants Final Evaluation Report

New York State Unified Court System

Title: The NYS Medical Liability Reform & Patient Safety Demonstration Project
Award Number: R18 HS19505-01
Principal Investigator: The Honorable Judy Harris Kluger, J.D. 
Award: $2,999,787
Period of Performance: 7/01/10–6/30/15

Goals

Through initiatives to promote safety culture, a communication and resolution program (CRP), and an expanded and enhanced judge-directed negotiation (JDN) program, this grant aimed to enhance the patient safety culture in participating hospitals, prevent and address adverse events, prevent the filing of cases through disclosure and early offer, expedite the movement of cases through the claims process, increase the number of settlements, and, over time, lower malpractice costs and premiums.

Although the project received a no-cost extension through June 30, 2015, the five hospitals ceased project participation on June 30, 2013.

Methods

The grantee implemented three program components in five medium and large academic medical centers in New York City. These five hospitals provide care to some of the city’s most economically disadvantaged and medically underserved populations.

The first component was hospital-based initiatives to promote a culture of patient safety. The grant required that each of the hospitals focus its attention on developing a culture of patient safety by, at a minimum, (1) conducting the AHRQ patient culture survey at the beginning and at the end of the grant, (2) enhancing adverse event reporting and response, and (3) providing communication training to general surgical staff using either TeamSTEPPS^® or a similar type of training package. (TeamSTEPPS^®—Team Strategies and Tools to Enhance Performance and Patient Safety—is an evidence-based teamwork system developed by AHRQ and the Department of Defense’s Patient Safety Program to improve communication among health care providers.) Each hospital used a variety of patient safety-related initiatives and interventions tailored to its own organization. Some examples of specific interventions include a refined surgical safety universal protocol, a practice guideline for managing obese surgical patients, and improvements in adverse event reporting systems. The grantee estimates that 5,000 hospital staff across the participating institutions received TeamSTEPPS^® training.

The second component was a communication and resolution program (CRP) in the general surgery departments at each site. Staff at all five sites received disclosure training developed in consultation with the Greater New York Hospital Association and the Madison Consulting Group that included presentation of didactic material, role-playing, and small-group exercises. Different hospitals had different types of staff attend the training (e.g., surgeons, other clinicians, risk managers, quality and patient services staff). Reportedly, participant reaction to the training was overwhelmingly positive. A 6-month follow-up satisfaction survey of training participants revealed that they found the skills they learned helpful and used them in practice. According to the grantee’s final report, the training continues to be provided to hospital staff by their malpractice insurer.

Each of the hospitals worked with the Department of Health staff to diagram how an adverse event involving severe harm would be managed, including the process they planned to use to make decisions about when to make a disclosure, the staff who would make it, and the process by which an offer and resolution would follow. Based on the University of Michigan Health System model (Boothman, Blackwell, Campbell, et al., 2009), the CRPs involved prompt reporting of the adverse event to the hospital, full disclosure to the patient/family, a timely investigation, an explanation of investigation findings and an appropriate apology to the patient/family, and an effort to learn about patient safety. In addition, when providers failed to meet the standard of care, CRPs were to seek to achieve a resolution by offering compensation without requiring families to sue. The project provided customized training in disclosure skills to clinicians at each site.

The third component of the grant was the enhanced judge-directed negotiation (JDN) program, which was modeled after a program developed in 2004 by the Honorable Douglas McKeon, New York State Supreme Court 12th Judicial District, Bronx County. Judge McKeon continues to provide leadership to the program. The JDN program aimed to prevent protracted legal battles and reduce the legal costs and frustration for providers and patients who pursue malpractice claims through the traditional court system. As a result of the grant, all malpractice lawsuits against the five participating hospitals were directed to a special program of judge-directed settlement negotiations. Under this model, a judge with experience in medical malpractice cases and training in clinical issues and negotiation skills presides over the malpractice lawsuit from the filing of the case to the end or until a trial is scheduled. The judge convenes the parties “early and often” to discuss the case and help broker an expedited and equitable settlement. The judge facilitates negotiations but does not attempt to impose a settlement or a settlement amount. The plaintiff may move ahead with trial if the parties do not agree on a settlement after a period of negotiation. One significant advantage of this approach to malpractice liability reform is that it does not require any change to the law.

With grant funding, the JDN program was expanded to include courts in Brooklyn and Manhattan, where additional training increased the number of judges to five. A full-time nurse-attorney was hired to provide input when the judge or lawyers needed clinical information, to keep notes on the cases and their disposition, and to enter data for the evaluation.

Analysis

Communication and resolution program. CRP process and outcomes were measured using structured data on CRP events entered by hospital staff, insurer data on the aggregate numbers of claims processed during the intervention period, surveys of participating physicians (for each case, the physician most involved in the case received a 15-item survey), and interviews with hospital administrators about their experiences with the CRP program. Over 22 months, starting in September 2011, data were collected on 125 CRP events. Surveys were received from 70 of the 123 physicians who received one.

Enhanced judge-directed negotiation program. The evaluation of the JDN program had two components. The first was a litigant satisfaction survey, in which opposing attorneys were asked to complete an online survey about how the JDN affected overall outcomes. The second evaluation component assessed the effect of the JDN on the resolution of cases. The focus was primarily on the time to disposition, percent of cases settled, and the stage at which cases were settled. It should be noted that the final analysis of data is not complete, as the grant project was extended through June 2015 in order to allow time for additional data collection and analysis.

From the beginning of the grantee’s data collection in September 2011 to the submission of its final report, 326 cases had entered the JDN and 32 had reached a final disposition. At the time of the final report, the grantee had received 16 responses from the 31 attorney surveys sent. The findings on the JDN presented below reflect the data analysis at the time of the grantee’s final report.

Findings

Communication and resolution program. Findings regarding the CRP implementation are below:

• Of the 125 CRP events, the greatest proportion resulted in temporary but severe harm (68.8%). Cases next often involved death (15.2%) and permanent but not severe harm (7.2%). In more than half of cases, patients required altered medical management (56.8%) and a prolonged hospital stay (52.0%), with 11.2 percent of injured patients acquiring a disability. The most common clinical events were unintended laceration or puncture (18.4%) and anastomotic leak/enterotomy (16.8%).
• Overall, the hospitals implemented the key steps of the CRP. An initial disclosure was documented in 92 percent of CRP cases. Communication that an adverse event occurred was documented in 79 percent of cases, with explanations of the reasons for harm (88%) and “apologies of sympathy” (64%). An apology acknowledging responsibility was given in only 13 percent of cases.
• A quarter (25%) of the events assessed were reported to Risk Management or Quality within 24 hours, half (50%) within 6 days, and three-quarters (75%) within 18 days, with only a short time elapsing between the first communication with patients and families about the event and the last communication (50% were concluded within 2 days). Overall, 75 percent of communications were concluded within 32 days.
• Of the 125 events, 30 were judged to involve “all of the essential elements of a negligence claim: an injury, causation, and substandard care” (Mello, 2015, p. 8). For these 30, the responses deemed appropriate included compensation (9) and an offer to waive medical bills (12).
• Very few CRP cases (20) progressed to a claim or lawsuit. Twelve to 15 months after CRP completion, 14 of the 125 cases (11.2%) resulted in a filed lawsuit and six (4.8%) others resulted in a claim but no filed lawsuit. Three of these cases (2.4%) were settled with a release of claims. Eleven of the 20 cases with a claim or lawsuit involved substandard care and causation, and the hospital offered compensation in 3 of these 11 cases.
• Physicians, nearly all of whom (98%) were surgeons, generally lacked awareness of the CRP. Most were not knowledgeable of the compensation given to the patient or the amount of time needed to resolve the case. However, physicians rated the CRP positively. On a scale of 1 to 100, physicians gave positive ratings regarding disclosure conversations with the patient/family, the assistance they received in preparing for the disclosure conversation, their interactions with CRP representatives (when they had them), and their treatment by CRP representatives (medians of 85.5, 87.0, 89.5, and 75.0, respectively). Respondents were less positive about the likelihood that the program helped them avoid a lawsuit (median of 50.0).
• The CRP process shed light on the causes of harm. In cases involving violations of the standard of care, the leading human factors contributing to errors were communication problems among providers (17 cases, 56.7%) and inattention (13 cases, 43.3%). In these cases with substandard care, most (86.7%) involved a failure of an individual provider, and half (50%) involved a system failure.

The grantee encountered a host of barriers to implementation of CRP, including risk management workload, malpractice insurer reticence (because of fear that disclosure would increase claims), and institutional resistance (hospitals unwilling to negotiate directly with patients and families). Further, because New York lacks legislation supporting disclosure and resolution, the tort system is very polarized, making it difficult to garner enthusiasm for early resolution.

The grantee also noted that the disclosure side of the CRP was implemented much more successfully than the compensation/resolution side. In most cases in which there was deviation from the standard of care, compensation was not pursued. However, the project was successful in changing the culture and mindset around communication.

Enhanced judge-directed negotiation program. From the beginning of the grantee’s data collection in September 2011 to the preparation of this report in late May 2015, 716 cases had entered the JDN and 165 had reached a final disposition. The grantee had received 98 responses from the 211 attorney surveys sent. (Surveys were sent when the JDN process was completed, even if a final disposition had not yet been reached. Some surveys had not yet been sent at the time of this report.).

When the grantee submitted a final report at the end of the original grant period, it provided the following interim data analysis although enrollment into the JDN was ongoing. The final analysis of data is not complete, because the JDN program continued to accrue cases through April 2015.

• Of the 326 cases that had entered the JDN by the end of the original grant period, most (more than 80%) involved serious, permanent harm or death.
• Of the 32 cases with a final disposition at that point, 15 were settled and 17 were voluntarily dismissed. Across these cases, the median time between filing and disposition of the case was 189 days.
• For 12 cases settled during conference, the median settlement was $237,000, and the median time from filing to disposition of the case was 8 days. For three cases settled outside of conference, the median settlement was $55,000, and the median time was 240 days.
• It appeared to judges that the participants in JDN cases were “highly engaged and working hard to reach resolution.” According to the grantee, several factors appeared to have promoted settlement: having early dialogue, a “high-exposure” case, and agreement on key facts. Unrealistic plaintiff expectations appear to have hindered settlement.
• Preliminary findings from the attorney surveys received to date were encouraging: 90 percent of respondents thought the JDN program had reduced their case’s litigation costs, 80 percent thought the JDN was a positive contribution to case resolution, and 90 percent were satisfied or very satisfied overall.

Participating hospitals were more enthusiastic about participating in the JDN program than they were in implementing the CRP or the patient safety initiatives. From early in the project, involved stakeholders showed their interest in alternatives to taking claims to trial. Attorneys on both sides—for the plaintiff and defense—could see the value. The New York court system is committed to the JDN program and has continued the program past the end of the grant period using internal court funding. Hospitals that did not participate in the grant have asked to join the program, particularly hospitals in the City and outer boroughs, where there are high malpractice costs and a high concentration of cases.

Selected Publications and Presentations Developed by Grantee

Publications and Manuscripts:

Mello MM. The medical liability climate: The calm between storms is the time for reforms. Bill of Health Blog, Harvard Law School/Petrie Flom Center for Health Law Policy, Biotechnology, and Bioethics. Oct. 31, 2014. Available at http://blogs.law.harvard.edu/billofhealth/2014/10/31/the-medical-liability-climate-the-calm-between-storms-is-the-time-for-reforms/.
Mello MM, Greenberg Y, Senecal SK, et al. Case outcomes in a communication-and-resolution program in New York hospitals. Manuscript submitted for publication. 2015.
Mello MM, Senecal SK, Kuznetsov Y, et al. Implementing hospital-based communication-and-resolution programs: Lessons learned in New York City. Health Aff. 2014;33(1):30-38. PMID: 24395932.
Mello MM, Studdert DS, Kachalia A. The medical liability climate and prospects for reform. JAMA. 2014;312(20):2146-2155.
Sage WM, Gallagher TH, Armstrong S, et al. How policy makers can smooth the way for communication-and-resolution programs. Health Aff (Millwood). 2014;33(1):11-19. PMID: 24395930.

Presentations:

Cohn J. Offer and disclosure programs, the New York model. American College of Surgeons, 2012 Medical Liability Reform Summit; 2012 Oct 19; Washington, DC.
Cohn J. New approaches to medical liability reform, the role of the state: Can government help states to embrace patient safety and reduce medical liability costs? National Academy for State Health Policy, 24th Annual State Health Policy Conference, New Directions in State Health Policy; 2011 Oct 5; Kansas City, MO.
Cohn J, Senecal S. Poster: The New York patient safety/medical liability reform demonstration project. Agency for Healthcare Research and Quality, 2012 Annual Conference, Moving Ahead: Leveraging Knowledge and Action to Improve Healthcare Quality, Medical Liability Reform and Patient Safety Initiative; 2012 Sep 10; Bethesda, MD.
Kluger JH. Innovative medical liability reform in action. American Medical Association, State Legislative Strategy Conference; 2012 Jan 5-7; Tucson, AZ.
Kluger JH. Innovations in addressing medical malpractice claims, Part I. Judge American Health Lawyers Association; 2012 May 4.
Kluger JH. Litigation Section of the CBMA and the Academy of Medicine of Cleveland and Northern Ohio, Medical Malpractice Issues for Attorneys and Physicians, Medical Specialty Court Initiatives. 2012 Apr 17; Cleveland, OH.
Kluger JH. Second meeting: The appropriateness of dimensions in health care, the judge directed negotiation program. 2011 Nov 11; Milan, Italy.
Mello MM. Medical liability reform: What’s on (and off) the table now? Chicagoland Healthcare Risk Management Association Annual Meeting; 2010; Chicago, IL.
Mello MM. New directions in medical liability reform. Columbia Law School Health Law Colloquium; 2010; New York, NY.
Mello MM. Medical liability reform: What’s on (and off) the table now? DuPage County Medical Society Annual Meeting; 2010; Oak Brook, IL.
Mello MM. New Directions in medical liability reform. Dana-Farber/Harvard Cancer Center Outcomes Research Program Seminar; 2010; Boston, MA.
Mello MM. Current directions in medical liability reform. Johns Hopkins Bloomberg School of Public Health; 2012; Baltimore, MD.
Mello MM. New directions in medical liability reform. Keynote Address, American College of Surgeons Medical Liability Summit; 2012; Washington, DC.
Mello MM. Hospital-led malpractice reform: Disclosure-and-offer programs in Massachusetts and beyond. Neurosurgery Grand Rounds, Massachusetts General Hospital; 2012; Boston, MA.
Mello MM. Disclosure of medical errors and medicolegal risk: Is it safe to be honest? Health Law Distinguished Speaker Series, Center for Health Law Studies, St. Louis University School of Law; 2012; St. Louis, MO.
Mello MM. Disclosure and resolution programs. Medical Malpractice Roundtable, University of Texas School of Law; 2013; Austin, TX.
Mello MM. Medical liability and accountability for safe care: Looking back, moving forward. Regis J. Fallon Lecture, Center for Health and the Social Sciences, University of Chicago; 2013; Chicago, IL.
Mello MM. Disclosure of medical errors: Is it safe to be honest? Ethics Grand Rounds, Dana Farber Cancer Institute; 2013; Boston, MA.
Mello MM. Next steps for communication and resolution programs. National Quality Colloquium Bootcamp; 2013; Washington, DC.
Mello MM. Disclosure and resolution programs as a quality improvement initiative. Third HealthPact Forum, 2013; Seattle, WA.
Mello MM. Implementing communication and resolution programs: Insights from early adopters. Georgia State University Center for Law, Health and Society Workshop; 2013; Atlanta, GA.
Mello MM. Implementing communication-and-resolution programs in hospitals. American Society for Healthcare Risk Management Annual Meeting; 2013; Austin, TX.
Mello MM. Managing medical liability risk through error disclosure and proactive compensation: Opportunities and challenges. Beth Israel Deaconess Medical Center Medicine Grand Rounds; 2014; Boston, MA.
Mello MM. Update on medical malpractice. Petrie-Flom Center Health Law Year in P/Review, Harvard Law School; 2014; Cambridge, MA.
Mello MM. Managing the risks of malpractice and error disclosure. Harvard Medical School Leadership Development for Physicians and Scientists Program; 2014; Boston, MA.
Mello MM. Hospitals’ responses to medical errors: The role of alternative dispute resolution. Quinnipiac-Yale Dispute Resolution Workshop; 2014; Hamden, CT.
Mello MM. Operation and effects of communication-and-resolution programs in hospitals. Empirical Health Law Conference, Cornell University; 2014; Ithaca, NY.
Mello MM. New directions in medical malpractice reform. Deinard Memorial Lecture on Law and Medicine, University of Minnesota; 2014; Minneapolis, MN.
Mello MM. Can communication-and-resolution programs help hospitals avoid malpractice suits? American Association of Law Schools Annual Meeting; 2015; Washington, DC.
Mello MM. New directions in medical liability reform. Department of Health Policy and Administration Workshop, Gillings School of Public Health, University of North Carolina at Chapel Hill; 2015; Chapel Hill, NC.

Selected Other Products Developed by Grantee

• CRP training package for clinicians and JDN training for judges
• Litigation satisfaction surveys (provider, patient, and attorney)

Source Documents for This Grantee Profile

Boothman RC, Blackwell AC, Campbell DA, et al. A better approach to medical malpractice claims? The University of Michigan experience. J Health Life Sci Law. 2009 Jan;2(2):125-159. PMID: 19288891.
Kluger JH. Improving Communication With Patients—NY State Unified Court System. Final Progress Report. Prepared by New York State Unified Court System under grant number R18 HS19505. Prepared for the Agency for Healthcare Research and Quality. New York, NY; Jun 2014.
McKeon D. New York’s innovative approach to medical malpractice. New Eng L Rev. 2011-2012;46(3):475.
Mello MM, Greenberg Y, Senecal SK, et al. Case outcomes in a communication-and-resolution program in New York hospitals. Manuscript submitted for publication. 2015.

University of Illinois at Chicago (UIC)

Title: The Seven Pillars: Crossing the Patient Safety-Medical Liability Chasm
Award Number: R18 HS19565-01
Principal Investigator: Timothy McDonald, M.D., J.D.
Award: $2,998,083
Period of Performance: 7/01/10–6/30/14

Goals

This project is evaluating the effectiveness of a comprehensive process for responding to patient harm events known as the “Seven Pillars” that has been in place at the University of Illinois Medical Center at Chicago (now part of the University of Illinois Hospital and Health Sciences System, UIHHSS) since April 2006. The long-term objectives of the project are to improve patient safety and mitigate medical liability risk through (1) improved communication with patients and families, (2) disclosure and early offer when patients suffer preventable harm, and (3) learning from medical errors.

The intent of the project was to allow UIC to further refine the Seven Pillars, replicate the program from its academic hospital setting to 10 diverse community hospitals in the greater Chicago area, and measure the impact at these hospitals, as well as to collect and report additional data on the impact of Seven Pillars at UIC.

Methods

The key components of the Seven Pillars process include the following:

1. Patient safety incident reporting
2. Investigation and root cause analysis
3. Communication and disclosure
4. Apology and remediation
5. Patient safety and systems improvement
6. System process and performance improvement
7. Education and training

The 10 Chicago-area community hospitals were randomized into two groups (early and late implementers) for purposes of the evaluation. The grantee conducted baseline gap analysis (described below) to measure each hospital’s implement readiness across various areas, and the initial training was intended to overcome the identified gaps. Three types of initial training were provided as part of the Seven Pillars implementation: (1) patient communication (crisis management and disclosure) training; (2) “care for the caregiver” training; and (3) resident training on reporting patient safety events. This initial set of trainings was complemented with follow-up training, “just-in-time” training, and as-needed training and consultation. The grantee noted that the roll-out of training was slower than expected, so that parts of the intervention were still being deployed in the five “early” implementing sites when the five “late” implementing sites came on board, one issue that may have made it difficult to differentiate outcomes between the two cohorts. The grantee also noted that some participating hospitals requested special training on all Seven Pillars elements for their resident physicians. Although these trainings were outside the scope of the grant, the request was an encouraging indicator of a change in safety culture at these hospitals.

Seven Pillars was implemented in varying degrees at each of the 10 community hospitals. As expected, the Seven Pillars intervention was enhanced during the course of the demonstration in areas such as resident reporting, care for the caregiver, root cause analysis, and crisis management, with the enhancements being well received by the sites. Even in the third year of the grant, refinements to the training were ongoing. Several educational modules and tools were developed by the grantee, including training on data collection and a 40-hour CME-accredited program about Seven Pillars.

During the course of the grant, 7 of the 10 participating hospitals merged under a single corporate structure. Although the merger was initially a problem for implementation, the grantee noted, it ultimately facilitated dissemination of the model. Seven Pillars had a lot of support at the corporate level; and as a result, the grantee continued to work with the 7 hospitals initially included in the grant, as well as 13 or 14 other hospitals under the same corporate umbrella.

Analysis

For each of the participating community hospitals, the project team completed a baseline gap analysis using onsite meetings, focus groups, and key informant interviews to gather data from a range of individuals from frontline staff to executives and members of the board of directors. The gap analysis was intended to measure each hospital’s readiness to implement the intervention across various areas. The grantee shared the findings of the baseline gap analysis with the community hospitals, then provided the initial training on Seven Pillars to overcome the identified gaps.

The evaluation plan for the grant included two assessments of the impact of the intervention at the participating community hospitals:

• Impact on patient safety process and outcome measures (e.g., number of adverse event reports, numbers of reported significant adverse events, patient safety culture, number and quality of disclosures)
• Impact on risk management and medical liability process and outcome measures (e.g., number of claims, time to settlement agreement, malpractice premiums, proportion of settlement received by patient or family).

The evaluation team planned to use an interrupted time series with a non-equivalent, no-treatment control group, so that five hospitals received the intervention first and the other five were initially the control sites, receiving the intervention at a later time.

The grantee also conducted an evaluation of the impact of Seven Pillars at UIC. The analysis involved an interrupted time series design with UIC, with itself as a historical control. Because the UIC site began implementation in 2006, sufficient data were available to examine the impact of Seven Pillars on measures of patient safety process, outcomes measures (e.g., changes in physician practices), and risk/liability measures.

Findings

Replication site readiness. The gap analysis performed at each hospital proved useful to implementation. The grantee noted, for example, how corporate representatives for seven of the hospitals (those that merged during the course of the demonstration) identified as the result of the gap analysis the need for a Chief Medical Officer to implement Seven Pillars so they could establish a rapid response to patient harm that included physician engagement.

Impact of Seven Pillars at the replication sites. The grantee presents no data on the impact of Seven Pillars at the 10 community hospitals where the intervention was replicated in large part due to a failure to obtain the relevant data.

• Patient safety process and outcome measures. A secure data repository was established for the participating community hospitals to submit safety data, and data were collected through fall 2013; however, only 8 of the 10 sites regularly submitted data. Motivating the participating hospitals to supply data was a challenge. Because the research design called for a blinded analysis, the evaluation team could not complete the main analysis of impact of the intervention on the 10 hospitals until all data were received. To the best of our knowledge, UIC has not completed the proposed comparative analysis.
• Risk management and medical liability measures. Obtaining data on risk management and liability outcomes proved to be a challenge as well, and the grantee made adjustments to its plan for collecting these data. According to the final report, after many attempts to retrieve data on claims, payments, and other liability outcomes, and in consultation with a third party intermediary analysis team for the medical liability outcomes, the evaluation team determined that the only way they would be successful at retrieving any medical liability data from the community hospitals was to request hospital “loss-runs” for the pre- and postintervention periods. At last account, the evaluation team had received only a few months of such data.

  The grantee noted that, due to the short timeframe of the grant period, the analysis would not have resulted in statistical evidence on liability outcomes. The most important outcome that came to light through formal and informal measurements was the change in safety culture at the replication hospitals. This change may improve safety outcomes over time, but no specific data are available to support this claim.

Impact of Seven Pillars at UIC. The primary source of data was the UIHHSS incident reporting system, which was used to access records for 2000 to 2012. The analysis included 637 onsite incidents. The grantee first compared the means of each outcome measure during the period before the intervention to those after the intervention. A comparison of means from the pre- to post-intervention period revealed increases in mean quarterly number of incident reports (p<.0001), patient communication consults (p<.0001), and peer reviews (p<.0001) and decreases in mean quarterly number of claims (p=.0005), lawsuits (p=.0003), legal fees and expenses (p=.003), settlement amounts (p=.005), and total liability costs (p=.001) (Lambert et al., 2014). Per claim examination reflects the same changes as the cumulative quarterly examination, with significant reduction in quarterly lawsuits per claim and legal fees and total liability costs per claim (p=.0005) (Lambert et al., 2014, p. 25). A significant reduction was also found in mean time to closure per claim (p<.0001). In addition, self-insurance costs, which were increasing prior to the intervention, were decreasing afterward; likewise, the self-insurance fund balance, which was negative and decreasing prior to the intervention, moved from a $30 million deficit to a $40 million surplus (Lambert et al., 2014, p. 22, 23, 50). According to the grantee, “Overall, when viewed in terms of both the absolute level and trends in important outcomes, the Seven Pillars intervention achieved its main goals” (Lambert et al., 2014, p. 24).

An analysis of the impact of Seven Pillars was also conducted on changes in physician practices (and service charges) related to patients presenting with chest pain, comparing practice at UIC to other non-federal general hospitals in the county that recorded at least 800 discharges per quarter. This analysis investigated the notion that a communication and resolution program may create a perceived reduction in the risk of lawsuits on the part of physicians, thus decreasing practices of defensive medicine (e.g., earlier discharge, fewer imaging studies and tests). The grantee accessed discharge records from January 2002 through December 2009 for patients admitted for chest pain. They compared 2,215 UIC records to 102,220 records from comparison hospitals selected through a propensity-score matching algorithm. The grantee found that UIC, compared with 43 other large and medium-sized hospitals in the same jurisdiction, achieved statistically significant reductions in the growth rate of charges for clinical laboratory and radiology services. However, the statistical models used did not directly test whether there was any reduction in the absolute level of charges for either clinical laboratory or radiology services in connection with the introduction of the Seven Pillars intervention. Further modeling and interpretive support is needed to answer that question.

Selected Publications and Presentations Developed by Grantee

Bell SK, Delbanco T, Anderson-Shaw L, et al. Accountability for medical error: Moving beyond blame to advocacy. Chest. 2011 Aug;140(2):519-526. PMID: 21813531.
Boike JR, Bortman JS, Radosta JM, et al. Patient safety event reporting expectation: Does it influence residents’ attitudes and reporting behaviors? J Patient Saf. 2013 Jun;9(2):59-67. PMID: 23697981.
Helmchen LA. A cure for defensive medicine? The effect of a “disclosure-and-offer” program on inpatient mortality, length of stay, and charges. Presented at AHRQ; 2011 Oct 17; Rockville, MD.
Helmchen LA. A cure for defensive medicine? The effect of a “disclosure-and-offer” program on inpatient mortality, length of stay, and charges. Presented at Center for Health Policy Research and Ethics, George Mason University; 2011 Nov 10; Fairfax, VA.
Helmchen LA, Richards MR, McDonald TB. Successful remediation of patient safety incidents: A tale of two medication errors. Health Care Manage Rev. 2011 Apr-Jun;36(2):114-123. PMID: 21317663.
McDonald TB. Alternative approaches in responding to medical errors. Trial. 2013 May;49(5) 34-41.
McDonald T, Centomani N, Mayer D, et al. The Seven Pillars Approach to Adverse Events: Effect on Patient Safety and Medical Liability. Poster presented at the 2011 AHRQ Annual Meeting; 2011 Sep 19; Bethesda, MD.

Selected Other Products Developed by Grantee

• Pilot-tested and refined Seven Pillars gap analysis toolkit
• List of the Seven Pillars components required for successful implementation
• Pilot-tested communication assessment survey and scoring mechanism to identify hospital employees who are the most effective communicators to participate in the patient communication consult service
• Educational modules and tools, including a 7-hour training program for the global trigger tool, a 40-hour CME-accredited program for education around the Seven Pillars process, several cases, standardized patient encounters, and training video vignettes
• Resident reporting handbook for all resident physicians and program directors
• Data dictionaries and templates for collecting safety and liability outcome measure data

Source Documents for This Grantee Profile

Helmchen, LA, McDonald TB, Lambert BL. Changes in physician practice patterns after implementation of a communication-and-resolution program. Manuscript in preparation. 2014.

Lambert B, Centomani N, Smith K, et al. The “Seven Pillars” response to patient safety incidents: Effects on medical liability processes and outcomes. Manuscript in preparation. 2014.

McDonald T. The Seven Pillars: Crossing the Patient Safety–Medical Liability Chasm. Final Progress Report. Prepared by University of Illinois at Chicago under grant number R18 HS019565. Prepared for the Agency for Healthcare Research and Quality. Chicago, IL; Jun 2013.

University of Texas (UT) Health Science Center

Title: The Texas Disclosure & Compensation Study: Best Practices for Improving Safety
Award Number: R18 HS19561-01
Principal Investigator: Eric Thomas, M.D., M.P.H. 
Award: $1,796,575
Period of Performance: 7/01/10–6/30/14

Goals

The project aimed to implement and evaluate a disclosure and compensation model and identify best practices for using disclosure to improve patient safety, with a focus on incorporating patient and family input into efforts to understand why errors occur.

Methods and Analysis

Implementation of a Disclosure and Apology Training. During the first year of the grant (2010), the grantee provided disclosure training to staff at six participating health institutions (four medical schools, one cancer center, and one health science center) across the UT System. The purpose of the training was to provide an overview of key underlying principles and key communication skills necessary for disclosure of unanticipated outcomes and to set the stage for hospital-based teams to build and implement a disclosure consultation team model adapted to their own institutions. According to the grantee, “Physicians chosen to attend training were respected leaders in their institutions who could serve as disclosure coaches to other physicians and could help educate students and residents about disclosure” (Etchegaray, Gallagher, Bell, et al., 2015, p. 5). Each of the six hospitals implemented its own intervention based on the disclosure training. Some of the UT System hospitals developed more established disclosure processes; in other UT System hospitals, the program was more “organic.”

Development of a “Culture of Disclosure” Survey. The grantee developed and tested a new “culture of disclosure” survey, which fills an existing gap for the patient safety field and provides a tool for institutions to assess their disclosure culture. Health care organizations and researchers can use this survey as a stand-alone measure or as an add-on to any existing safety culture survey. The survey measures disclosure culture over time and identifies barriers to disclosure. It contains four scales: error disclosure general culture, error disclosure trust culture, safety, and teamwork, and also includes items on “intent to disclose” a hypothetical error. The items in the tool include those from a previous study conducted by the researchers as well as another instrument, the Safety Attitudes Questionnaire.

Before and after the disclosure training (2010 and 2013), the grantee administered the new culture of disclosure survey to all clinical faculty in the six participating institutions. The survey was completed by 496 and 1,217 clinical faculty before and after the training, respectively. Each participant was randomly given one of two hypothetical scenarios depicting a medical error of comparable severity and was asked questions about the assigned scenario. The grantee analyzed one of the scenario response questions: “How likely would you be to disclose this error to the patient?” A factor analysis was also completed to determine how many scales within the survey were needed to measure error disclosure.

Incorporating Patient Experiences and Perspectives. Another significant component of the grant was the exploration of ways to involve patients/families who have experienced an adverse event or a near miss in the patient safety improvement processes (e.g., event analyses) in health care organizations. Several methods were used to explore these issues. The first step was to conduct structured interviews with stakeholders about the disclosure process at the six participating UT System hospitals. Stakeholders included 6 clinicians, 13 hospital administrators, and 5 patients and 4 family members involved in an adverse event at the hospital. The researchers developed two sets of interview questions: one for clinicians and administrators and one for patients and family members.

Researchers shared the findings from the structured interviews with experts in disclosure and patient and family engagement at a national conference held by project leaders in 2011. These participants consisted of 5 risk managers or hospital administrators, 3 nurse clinicians, 3 patient safety experts, 3 patients, one frontline clinician, and 10 members of the multidisciplinary research team. Participants discussed aspects of “involving patients as partners” after an adverse event, including establishing the needs of patients and clinicians in the post-event period, contextual factors, and barriers (e.g., fear of legal action). The experts voiced some concern about the level and type of information that patients and families could provide, perhaps due to a lack of understanding of hospital workflow processes and of the event itself. Rather than establishing best practices for involving patients and family members as was originally planned, the grantee decided to examine the following questions: What can patients (and their family members) report about adverse events, and what value could be added by including them in the analysis of their own adverse event?

From August 2012 to July 2014, the grantee conducted structured interviews with 72 participants who had experienced medical harm after an adverse event and disclosure of that event. These participants were recruited by risk managers from 20 hospitals or heard about the study from one of three patient advocacy groups. The interviews, which ranged from 30 minutes to 2 hours, aimed to determine whether patients and family members were aware of contributing factors to the event, how their description of the event differed from the findings of the root cause analysis, and how they would want to be involved after the event. Transcriptions of the audiotaped interviews were used to conduct content analysis. Although hospital leaders were generally supportive, it was difficult for risk managers and other UT staff to find the time to identify and contact UT patients for interviews as originally planned. The project had to expand its search for additional patients beyond the participating UT System, and most of the interviews were conducted with patients treated outside the system.

Using the structured interview guide as a starting point and information gleaned from the parents and family members through the structured interviews, the grantee aimed to develop a new tool, the Improving Post-event Analysis and Communication Together (IMPACT) tool. The intent was for this tool to be used to gather information from patients and family members closely after the occurrence of adverse events.

Malpractice claims. Lastly, the grant aimed to assess malpractice claims against the six institutions within the UT System. The UT System self-ensures claims against all the health providers who practice and train at these sites. The grantee reviewed records on a total of 715 malpractice claims closed over 5 years: 2001–2002 (244 claims, 59 settlements), 2006–2007 (142 claims, 24 settlements), 2009–2010 (100 claims, 17 settlements), 2010–2011 (114 claims, 24 settlements), and 2011–2012 (115 claims, 24 settlements). According to the grantee, the analysis aimed to establish a baseline level of legal claims and outcomes and to develop an approach that could be used to assess the effects of disclosure on legal claims and outcomes over time. Importantly, several changes to the landscape occurred within the years in which claims were evaluated. In 2003, the state passed tort reform that established a cap for non-economic damages (“pain and suffering”) and a statute of limitations for claims. Further, the UT System made a system-wide commitment to the disclosure of medical errors in 2008 and operationalized that commitment in 2010 with the work of the grant. The analysis examined several claim-related outcomes. For example, the number of claims, lawsuits, and paid claims differed by year.  All of these measures dropped after Texas enacted malpractice reform.

Findings

Culture of Disclosure Survey. In a manuscript for publication, the evaluators report significant improvements from pre- to posttest on the survey for minor error disclosure, serious error disclosure, safety culture, and teamwork culture (all p<.05) (Etchegaray, Gallagher, Bell, et al., 2015)—but not for error disclosure trust culture (potential for loss of patient and peer trust in physician disclosing errors). In addition, at posttest, the 472 individuals who had participated in error disclosure training had significantly better perceptions of their work culture across all five domains (p<.001 for all analyses) than the 599 who had not. The grantee also examined whether the culture domains correlated with an individual’s intent to disclose a hypothetical error. For those having received training, minor error disclosure culture, serious error disclosure culture, and error disclosure trust culture were significantly associated with intent to disclose (p<.05 for all analyses), whereas all five culture domains were significantly correlated with intent to disclose among those not receiving training (p<.05 for all analyses). However, it is difficult to assess the strength and meaning of these findings with regard to the impact of the disclosure training at UT System hospitals, in part because it is not clear from the manuscript exactly how some of the statistical comparisons were constructed. The grantee also noted that low response rate to the survey limits the generalizability of the findings.

Structured Interviews With UT System Stakeholders. The stakeholder discussions through the structured interviews yielded several important findings: (1) Clinicians and administrators endorsed the current best practices for analyzing events and the goal of improving transparency and the safety culture, (2) all stakeholders agreed that patients and family members had not previously been involved in the analysis of events, and (3) patients and family members and many clinicians and administrators believed patients and family members should be involved in the event analysis process. Clinicians and administrators were unsure how to involve patients and family members, however, due to their lack of familiarity with hospital processes and medical/technical terms.

Interviews with Patients and Family Members Who Had Experienced Medical Harm After Adverse Event. Of the 72 participants, 37 were patients and 35 were family members. The relevant events primarily involved infection (25%), medication errors 18(%), or diagnostic errors 15(%) and resulted in permanent harm (39%) or death (36%). The discussions with patients and family members yielded a wealth of knowledge about their view of the disclosure and event analysis processes. Notably, most patients would like to participate in the event analysis process and be asked for their perspectives, but the ideal timing of that involvement might vary by individual and the nature and severity of the event. Further, patients may have different needs regarding the ideal person to conduct debriefing and the number of opportunities for debriefing. In addition, most respondents had no intention of pursuing legal action but felt in some cases it was the only way to access information about what happened to them. About half of participants indicated they would prefer to engage in event analysis through a structured interview format. Reasons for wanting to contribute to the event analysis included “1) to help those involved gain a deeper understanding of what happened and identify the causes of the event; 2) to present recommendations which might prevent future patients from harm and provide the hospital with insight about quality issues; and 3) provide emotional healing and post-event support for patients and families” (Ottosen, Etchegaray, Sedlock, et al., 2015, p. 7).

The grantee developed the IMPACT tool after removing irrelevant, redundant, and confusing items from the structured interview guide. Feedback was provided via teleconference by four of seven expert risk managers who were given the tool to review. They agreed it was a “useful way to engage patients and families in a conversation about their events” (Ottosen, Etchegaray, Sedlock, et al., 2015, p. 10) but expressed concern about discussing specific details about the factors that contributed to adverse events with patients and family members. The IMPACT tool could be useful for engaging patients and families after an adverse event, possibly revealing new information about the event and allowing those involved a chance to heal by expressing their feelings (Ottosen, Etchegaray, Sedlock, et al., 2015).

Malpractice Claims. Of the 715 claims reviewed, 148 cases (21%) resulted in a payment, with the remaining being closed without payment. All 148 payments were reached through settlement, none through trial judgment. Across all claims, cases most commonly involved treatment (35%) and surgery (30%), with the most common allegations being improper performance of procedure (13%) and failure to adequately treat (11%). An analysis comparing data from before (2001–2002) with after (2009–2012) tort reform revealed the following:

• The proportion of claims resulting in settlements did not change significantly, but the proportion of claims in which lawsuits were filed decreased (54% vs. 31%; p<.001).
• The proportion of claims that were dismissed decreased (34% vs. 22%; p=.001), but the proportion of claims that were closed due to an expired statute of limitations increased (32% vs. 51%; p<.001).
• Mean payments decreased from $290,992 to $90,429 (p<.001).
• Settlements reached after tort reform were significantly more likely than those before tort reform to prohibit disclosure. Across all years, in a subset of relevant settlement agreements (124), the great majority (89%) included nondisclosure agreements.
• The time from the event to a claim decreased, but there was no significant change in the time from a claim to closure.

Selected Publications and Presentations Developed by Grantee

Etchegaray JM, Gallagher TH, Bell SK, et al. Error disclosure: A new domain for assessing safety culture. Presented at the 2011 Agency for Healthcare Research and Quality Annual Conference; 2011 Sep 19; Bethesda, MD.
Etchegaray JM, Gallagher TH, Bell SK, et al. Error disclosure: A new domain for safety culture assessment. BMJ Qual Saf. 2012 Jul;21(7):594-599. PMID: 22562878.
Etchegaray JM, Ottosen MJ, Aigbe A, et al. Patients’ knowledge about the factors that contributed to their own unexpected events. Poster presented at the 2014 Annual Research Meeting of Academy Health; 2014 June 8; San Diego, CA.
Etchegaray JM, Ottosen MJ, Aigbe A, et al. Understanding reasons that clinicians do not speak up. Poster presented at the 2014 Annual Research Meeting of Academy Health; 2014 June 8; San Diego, CA.
Etchegaray JM, Ottosen MJ, Burress L, et al. Structuring patient and family involvement in medical error event disclosure and analysis. Health Aff. 2014 Jan;33(1):46-52. PMID: 24395934.
Ottosen M, Etchegaray J, Burress L, et al. Learning from errors through patients and families. Poster presented at the 2012 Summer Institutes on Evidence-Based Quality Improvement; 2012 Jul 19-21; San Antonio, TX.
Sage WM, Gallagher T, Armstrong S, et al. How policymakers can smooth the way for communication-and-resolution programs. Health Aff. 2014 Jan;33(1):11-19. PMID: 24395930.

Selected Other Products Developed by Grantee

• Patient-centered interview tool, IMproving Post-event Analysis and Communication Together, (IMPACT) for eliciting patient and family perspectives on their harmful events
• Survey to measure both safety culture and error disclosure culture
• Institutional Reference Guide, with recommendations for patient and family partnership in postevent analysis

Source Documents for This Grantee Profile

Etchegaray JM, Gallagher TH, Bell SK, et al. The Error Disclosure Culture Survey and its implications for organizational culture. Manuscript submitted for publication. 2015.
Ottosen MJ, Etchegaray JM, Sedlock EW, et al. Developing the Improving Post-event Analysis and Communication Together (IMPACT) tool to involve patients and families in post-event analysis. Manuscript submitted for publication. 2015.
Sage WM, Dougherty JR, Colvard MJ, et al. The far side of tort reform: Experience and opportunities for improving resolution of malpractice claims against physicians in a public academic health system. Manuscript submitted for publication. 2015.
Thomas E, Gallagher T, Etchegaray J, et al. The Texas Disclosure and Compensation Study: Best Practices for Improving Safety. Final Progress Report. Prepared by University of Texas Health Science at Houston under grant number R18 HS019561. Prepared for the Agency for Healthcare Research and Quality. Houston, TX; Jun 2014.

University of Washington

Title: Communication to Prevent and Respond to Medical Injuries: WA State Collaborative
Award Number: R18 HS19531-01
Principal Investigator: Thomas H. Gallagher, M.D.
Award: $2,972,209
Period of Performance: 7/01/10–6/30/14

Goals

This grantee conducted a statewide initiative involving many stakeholders, including hospitals, multi-specialty physician groups, integrated healthcare delivery systems, professional associations representing the fields of medicine, nursing, and pharmacy, patient advocates, regulators, and an external insurer. The grantee aimed to implement a set of interventions to prevent and respond to medical injuries with adverse event analysis, disclosure, and compensation. It tested whether these “open” hospital systems and an insurer for physicians and clinics could collaborate on communication and resolution and whether these interventions would affect patient safety and liability costs.

Methods

The University of Washington implemented and evaluated three interventions to improve patient safety and reduce medical liability costs. The first intervention was the development of a diverse statewide collaborative, HealthPact, which included attorneys; State legislators; patient advocates; and representation from health care systems, insurance companies, medical and related associations, and the State board of medicine. The HealthPact collaborative hosted several statewide meetings that addressed communication and accountability in health care and established an information-sharing and skill-building community.

One group formed as part of HealthPact—the Patient and Family Advisory Council (PFAC)— developed, tested, and modified a simulation to heighten stakeholder awareness about the experience of patients and family members following adverse events. The simulation, which involves a patient whose cancer diagnosis is delayed as a result of miscommunication among her providers, takes the audience from the disclosure to the patient through the patient’s experience with the hospital, attorneys, and the medical board. The grantee implemented the interactive 2-hour exercise at two 100-person meetings, each time seating diverse stakeholders (e.g., patient advisors, clinicians, administrators, risk managers, insurers, patient safety experts, attorneys, regulators) together at each table to discuss the scenario as groups at various times throughout the activity. A narrator led the exercise, which also included an actor as the patient and two trained professionals. In demonstrating breakdowns in the response to medical injury, the exercise helped participants understand the perspectives and needs of injured patients and family members in new ways and revealed challenges they face. The grantee suggested some solutions, such as new ways of providing support to and representation of patients and family members, as well as possible adaptations and guidance for other groups interested in using this tool.

The grantee noted, “Participants considered the session to be extremely valuable and unlike anything they had encountered, largely due to the pervasive influence of the patient and family advisors. Not only did the PFAC design the entire exercise, their presence at each table helped surface barriers to responding to medical injury that stakeholders were unaware of. More importantly, their presence forced stakeholders to come face-to-face with the suffering patients and families experience from gaps in the response to medical injury, problems that stakeholders understood intellectually but perhaps discounted their significance. For many stakeholders, these interactions with the patient and family advisors heightened their motivation to seek novel solutions” (Gallagher, Etchegaray, Bergstedt, et al., 2015, p. 12).

The second intervention aimed to prevent and respond to medical injuries through enhanced, sustained communication practices at 5 hospitals and 3 large multi-specialty provider groups. This intervention included two components:

• Team communication training (including the additional train-the-trainer sessions), which aimed to create and support a small group of local trainers at each site to implement team communication training largely based on TeamSTEPPS (3-13 trainers from each site were trained). Prospective trainers attended a 2.5-day master TeamSTEPPS training, participated in a 4-hour training of trainers, and conducted their first onsite communication training with observation by the grantee’s trainers. Trainers were selected to participate primarily from two or three units identified as “early adopters.” They included opinion leaders across professions who were comfortable with teaching the team communication concepts and were involved with patient safety initiatives.
• Error disclosure training, which aimed to create a group of disclosure coaches at each site who could provide frontline clinicians with guidance and just-in-time coaching on error disclosure in partnership with risk managers (20–40 coaches from each site were trained). Prospective trainers participated in a tailored 4-hour, onsite training and monthly follow-up calls. Selected coaches were individuals with strong communication skills, a visible position, knowledge of local policies, and respect of peers, and they included a mix of representation (e.g., clinicians, risk managers, medical directors).

Training was developed using best practices from education theory: adult learning theory, Bloom’s taxonomy, interprofessional education for collaborative practice, and Kotter’s stages of change model. Training involved interactive sessions and skill building. Notably, the initial intent for the communication training component was to conduct a 4-hour training at each site. Subsequently, a train-the-trainer model was adopted; this approach proved beneficial to building confidence and enhancing the teaching skills of the onsite trainers but, as a significant expansion of the scope of the project, put enormous stress on the grantee and the sites. Among the lessons learned, the grantee noted that its conceptual model (“transparent communication, before and after preventable adverse events, as a unified whole with patient-centered care at the core”) was common to both trainings and could be linked to the sites’ mission, increasing buy-in from leaders and participants.

For the team communication training, 71 participants have fulfilled the Master TeamSTEPPS training, 69 participants have completed the train-the-trainer module, and over 1,300 providers have completed the front-line clinician training. Almost 400 disclosure and apology coaches have been trained.

The third intervention was the development, implementation, and evaluation of a communication and resolution program (CRP), formerly called a disclosure and resolution program (DRP), with 5 hospitals and Physicians Insurance (a provider of medical professional liability insurance for physicians and clinics). CRP entails early investigation and enhanced communication between the health care team and patient after an adverse event. The program was based on the model used by the University of Michigan Health System (UMHS). According to the grantee, “The crux of the program is that health care providers who injure a patient should report the incident promptly, disclose it to the patient or family, initiate and carry out a timely investigation of why the injury occurred, feedback investigation findings to the patient/family, and proactively seek an appropriate resolution, which may include financial compensation” (Gallagher, Shannon, Brock, et al., 2014, p. 14). The grant’s work was unique in that it attempted to apply the UMHS model, used primarily in closed systems (self-insured hospitals), in an open system (in which “facility and physicians carry separate insurance and most physicians are not employed by the hospital.” (Gallagher, Shannon, Brock, et al., 2014, p. 14).

The CRP model differed from the UMHS model in some ways. Notably, all incidents of unanticipated harm were eligible for the CRP, not only cases of serious harm, and CRP partner organizations were expected to collaborate on the analysis of events, disclosure, and compensation. Participating sites received an implementation toolkit, training, readiness assessment with feedback, and ongoing coaching.

CRP Certification pilot. In an effort to encourage disclosures, HealthPact has planned to pilot a novel approach called CRP Certification in partnership with Washington State’s Medical Quality Assurance Commission (MQAC). This approach responds to a commonly reported barrier to participation in CRPs: Clinicians fear that reporting adverse events may result in punitive consequences by regulators such as state medical boards. The grantee has been planning the pilot program for 24 months and aims to begin the 18-month implementation beginning in July 2015. Under the program, physicians and institutions that participate in CRP following an error can document the CRP process in a CRP Certification application and submit it to the Foundation for Health Care Quality. A review panel consisting of at least two physicians, a risk or claims manager, a quality improvement expert, and two patient advocates reviews the application. If members unanimously agree that the case meets all the established CRP requirements, the case is deemed “CRP certified.” The applicant can submit the review panel’s report to MQAC, which intends to close CRP-certified cases as satisfactorily resolved without further investigation. The grantee noted a few challenges it has encountered in the planning process, particularly those related to concerns about accountability to and protection of the public. The grantee will evaluate the pilot program using a set of key metrics.

Analysis

HealthPact Leadership Group. One-on-one interviews were conducted with members of the HealthPact Leadership Group, first in 2012 and then again in 2014.

Team communication training and disclosure and apology coach training. The grantee conducted a multifaceted evaluation of the trainings that included the following:

• Qualitative interviews, 3 with organizational leaders and 10 with participants in the disclosure and apology coach training.
• Training evaluations completed by 159 of 251 participants in disclosure and apology coach training. The instrument measured content, training, instruction, and efficacy.
• Attitude surveys (including the Organizational Change Scale and Disclosure Culture Scale), completed by 85 of 251 participants in disclosure and apology coach training.
• Training satisfaction for team communication training and disclosure and apology coach training using a paper-and-pencil instrument.

Communication and resolution program. Plans to evaluate the CRP included the following components:

• Semistructured telephone interviews with organization leaders and risk and claims managers before and after the interventions—coupled with information gathered through meetings with partner sites and data on incidents—to measure the extent to which policies, structures, and processes were changed.
• Analysis of claims data to measure costs and related liability outcomes. Data on each CRP case were to be entered by risk and claims managers on an ongoing basis into a customized REDCap data entry form.
• Satisfaction surveys for patients/family members and physicians who had cases undergo the CRP process.

Assessment of rates of communication-sensitive adverse events. In addition to these evaluations, the grantee initiated a novel assessment of the rates of communication-sensitive adverse events (CSAEs) and quality metric adherence at participating intervention hospitals and control hospitals. A CSAE was defined as “an adverse clinical event for which a lack of adequate communication may have been a contributing factor” (Slade, Kramer, Beck, et al., 2015, p. 5). HealthPact experts reviewed a list of national clinical quality metrics drawn from several sources and identified CSAEs, the metrics for which communication failure could have been a factor. Two sets of state data were obtained to conduct retrospective analyses—administrative data from the Comprehensive Hospital Abstract Reporting System (CHARS) and clinical registry data from the Surgical Care and Outcomes Assessment Program (SCOAP)—for the baseline period of 2009–2011 and the post-intervention period (2013).

Findings

HealthPact Leadership Group. As reported in interviews with members of the group, members found their initial goals for HealthPact to be lofty (e.g., “transform communication in health care”). However, most agreed that “the forums successfully increased the visibility of the issues around communication by bringing together diverse populations of concerned stakeholders, including adversaries (the plaintiff’s bar and the defense bar, and to some extent competing hospitals and physicians) to discuss disclosure and resolution” (Gallagher, Shannon, Brock, et al., 2014, p. 4).

Team communication training and disclosure and apology coach training.

• Qualitative interviews. Themes emerging from the interviews “can be summarized as increased need for organizational support, questions regarding what constitutes adequate training, resources available to support sustainability of efforts, continuing uncertainty regarding best-practice management of error, accounting for the… [different perspectives] of coaches from different disciplines, and innovation in application and support of ongoing error disclosure training” (Gallagher, Shannon, Brock, et al., 2014, p. 9).
• Evaluations of disclosure and apology coach training. The training was well received, with ratings in each item and subscale being high.
• Attitude surveys (including the Organizational Change Scale and Disclosure Culture Scale). In combination with the interviews, the Organizational Change Scale showed that respondents were enthusiastic about the training but lacked confidence that stakeholders would effect real change in the near future. The Disclosure Culture Scale revealed respondents’ general agreement that there was support for disclosure of error to patients, but their responses were less in agreement about whether adequate training on disclosure was provided or whether retaliation, loss of trust, or damaged reputations were concerns.

The grantee noted that some components of the planned evaluation could not be implemented for a variety of reasons (e.g., length of time to meet IRB requirements, time required of participants to complete tools, lack of validated instruments) and argued that developmental evaluation may be a more appropriate approach in future studies like this one.

CSAEs. Although the analysis will assess differences between CSAEs over time (before vs. after implementation of the communication training), only baseline data are currently available. CHARS data from 2009 to 2011 included 4 implementation sites and 93 comparison sites, and SCOAP data from this period included the same 4 implementation sites as well as 48 comparison sites. The CHARS dataset contained 1,684,916 unique patients, including 211,593 who experienced a CSAE. The SCOAP dataset contained 51,537 unique patients, including 5,004 who experienced a CSAE. The baseline rates for individual CSAEs were generally very low, <0.001% for CHARS and <1% for SCOAP, with most rate ratios favoring HealthPact hospitals over comparison sites. The grantee speculated that CSAEs are underreported.

Communication and Resolution Program.

• Semistructured telephone interviews with organization leaders and risk and claims managers before and after the interventions—coupled with information gathered through meetings with partner sites and data on incidents. Overall, the participating sites were not able to implement the CRP as intended. The grantee’s final report presented a number of challenges identified in interviews and observations: “reluctance to be in the vanguard,” “practical constraints arising from the liability insurer’s distance from the point of care,” “delays in incident reporting,” “lack of a clear implementation plan with assigned roles and responsibilities,” “overcoming distrust and missteps,” “risk managers’ and claim managers’ heavy workloads,” “persistent focus on serious-harm events,” “major disruptions and distractions,” and “uneven support for the CRP among key personnel in the organization” (Gallagher, Shannon, Brock, et al., 2014, pp. 17-19).
• Analysis of claims data to measure costs and related liability outcomes. Due to the selective use of the CRP process and risk and claims managers not using REDCap to track cases for a number of reasons (e.g., time needed to gather the information), the grantee collected data on only 30 CRP cases, an insufficient number to draw any conclusions about the effect of the CRP on liability.

The grantee concluded that “operating CRPs in which two or more insurers must collaborate to resolve cases is highly challenging and likely requires several preconditions not present for our sites, including a commitment from physicians to collaborate with facilities to resolve incidents, mechanisms for quickly transmitting information to remote insurers, tolerance for missteps in early attempts at collaboration, and clear protocols for joint investigations and resolutions” (Gallagher, Shannon, Brock, et al., 2014, pp. 17-18).

Selected Publications and Presentations Developed by Grantee

Bell SK, Delbanco T, Anderson-Shaw L, et al. Accountability for medical error: Moving beyond blame to advocacy. Chest. 2011 Aug;140:519-526. PMID: 21813531.
Brown SD, Lehman CD, Truog RD, et al. Stepping out further from the shadows: Disclosure of harmful radiologic errors to patients. Radiology. 2012 Feb:262(2);381-386. PMID: 22282177.
Etchegaray JM, Gallagher TH, Bell SK, et al. Error disclosure: A new domain for safety culture assessment. BMJ Qual Saf. 2012 Jul;21(7);594-599. PMID: 22562878.
Gallagher TH, Levinson W. Physicians with multiple patient complaints: Ending our silence. BMJ Qual Saf. 2013 Jul;22(7)521-524. PMID: 23576772.
Lu D, Guenther E, Wesley AK, et al. Disclosure of harmful medical errors in pre-hospital care. Ann Emerg Med. 2013 Feb;61(2):215-221. PMID: 22883681.
Sage WM, Gallagher TH, Armstrong S, et al. How policy makers can smooth the way for communication-and-resolution programs. Health Affairs. 2014 Jan:33(1);11-19. PMID: 24395930.

Selected Other Products Developed by Grantee

• HealthPact Web site, http://www.healthpact.org/, that includes HealthPact materials used for team communication training, disclosure coach training, and Communication and Resolution Program (e.g., implementation, training, and evaluation material)

Source Documents for This Grantee Profile

Gallagher TH, Etchegaray J, Bergstedt B, et al. Improving stakeholder understanding of accountability following medical injury using a patient-created simulation exercise. Manuscript submitted for publication. 2015.
Gallagher TH, Farrell M, Karson H, et al. Collaboration with regulators to support quality and accountability following medical errors: The CRP certification pilot. Manuscript submitted for publication. 2015.
Gallagher TH, Shannon S, Brock D, et al. Communication To Prevent and Respond to Medical Injuries: WA State Collaborative. Final Progress Report. Prepared by University of Washington under grant number R18 HS019531. Prepared for the Agency for Healthcare Research and Quality. Seattle, WA; Jun 2014.
Slade IR, Kramer CB, Beck SJ, et al. Communication-sensitive adverse events and quality metrics: A retrospective database analysis in Washington State hospitals. Manuscript submitted for publication. 2015.

Ascension Health

Title: Healing without Harm: A Multi-Site Demonstration Project to Develop New Models for Medical Liability and Improve Patient Safety
Award Number: R18 HS19608-01
Principal Investigator: Ann Hendrich, Ph.D., R.N., F.A.A.N.  
Award: $2,990,612
Period of Performance: 7/01/10–6/30/13

Goal

This project focused on ways of improving both the quality of perinatal patient care delivery and the management of adverse perinatal events in five geographically dispersed hospitals to decrease occurrences of birth trauma and reduce resulting medical claims. The project established a uniform, evidence-based obstetrics practice model for dealing with shoulder dystocia based on the idea that eliminating variation in obstetrics practice would translate to improved patient safety. The project also aimed to identify risk factors for shoulder dystocia.

Methods

The grantee recruited five Ascension Health hospitals to implement both clinical and coordinated communication protocols based in part on elements of a previously tested perinatal safety initiative, Handling All Neonatal Deliveries Safely (HANDS). The geographically dispersed hospitals were diverse in patient demographics, birth volume, and other factors.

Training. The clinical interventions targeted the “most injurious deliveries” by training physicians and nurses in labor and delivery units about electronic fetal monitoring (EFM) and the management of shoulder dystocia with the goal of improving the response to obstetric emergency.

• Training on fetal monitoring included best practices in EFM use, communication of readings, and interpretation and response to readings.
• Clinicians received didactic education in shoulder dystocia management (some of which was done through an Elearning module), TeamSTEPPS^® teamwork and communication training, and interdisciplinary simulation training using a high-fidelity simulator, the Noelle mannequin.

All 76 clinicians serving on an Obstetrics Event Response Team (OBERT) received training on implementation of the high reliability cause analysis and quick response model, 93% of 302 clinicians completed the training on disclosure and documentation of unintended events, and 90% of clinicians completed the shoulder dystocia simulation training and Elearning module as well as an Elearning module on fetal assessment and monitoring (Santos, Ritter, Hefele, et al., 2015).

Shoulder dystocia policy and bundle implementation. The shoulder dystocia management plan included the adoption of the American Congress of Obstetricians and Gynecologists (ACOG) definition of shoulder dystocia; development of new guidelines; simulation training in handling shoulder dystocia events using a new bundle^h that focused on improving situational awareness and communication, particularly the initial call-out of the event; and improving documentation of these events. The project team used the principles and practices of a High Reliability Organization (HRO) to facilitate adoption and spread of these standardized practices. The guidelines were developed by the Ascension Health perinatal steering committee, representing obstetricians, labor and delivery nurses, and a national expert. The steering committee followed the Institute of Medicine’s clinical guideline development standards. The process included an environmental scan (collection of information through peer-reviewed literature, unpublished documents, and interviews) and the sharing of meeting minutes to ensure a transparent process.

The shoulder dystocia guidelines call for the following steps during delivery (obstetric teams use digital timers and a shoulder dystocia delivery note for tracking):

• Record the delivery time of the head
• Call out shoulder dystocia if present, with documentation of the time of the call-out
• Attempt maneuvers to deliver the shoulders, with documentation of the sequence of maneuvers
• Announce  elapsed time using 30 second intervals  following call-out of shoulder dystocia to maintain situational awareness
• Document the delivery time of the shoulders
• If the shoulders are not delivered within 3 minutes of the head, prepare for  emergent abdominal rescue

The shoulder dystocia bundle was implemented from July 2010 through June 2011, and data were collected for this year and the 2 following years.

Each of the five sites implemented the shoulder dystocia program using a practice engagement team that included an obstetrical practitioner principal investigator, obstetrics nurse lead, risk manager, project manager, and medical coder. This group managed implementation, provided training to the obstetrics staff, shared project materials, and communicated with other sites to share experiences and improve the program over time.

Shoulder dystocia risk factors. As a separate grant activity, the grantee enrolled pregnant women in a substudy to identify demographic and clinical risk factors for a delivery involving shoulder dystocia. The substudy required extensive data collection on hundreds of demographic and clinical variables from participating mothers.

Communication intervention aimed at reducing harm. The Rapid Report Investigate and Disclosure Intervention, included processes for addressing all harm events, including “immediate identification and reporting of events that result in patient harm; uniform and expedited investigation and analysis of root and common causes of obstetrical events; prompt, transparent, and ongoing disclosure; and quick resolution of events involving probable liability” (Hendrich, McKoy, Gale, et al., 2013, p. 8). An Obstetrics Event Response Team (OBERT) was established at each site and was trained in emergency response, communication with patients and families, and conduct of apparent and root cause analysis. This team determined when an error needed to be elevated to the resolution stage.

Analysis

The grant aimed to assess the effects of the interventions on the application of clinical protocols, injury rates and infant harm, reporting of serious safety events, and severity of claims and settlement amounts. A case study methodology was used to assess outcomes before and after implementation of the model, which was fully implemented by April 2011. Outcomes were compared for the 15 months before implementation (January 1, 2010–March 31, 2011) with the 27 months afterward (April 1, 2011–June 30, 2013). The project team solicited extensive feedback around the education and training program. Data were collected through interviews with team members and document review. Practice data were collected for 3 years from the five participating demonstration sites implementing the shoulder dystocia bundle. A regression model was used to control for the large number of demographic and clinical variables in the quantitative analysis of the shoulder dystocia risk factor data.

Findings

Training. A key finding is that most participants value and are willing to participate in training, but the training has to be relevant and accessible. Clinicians reported that the training increased their confidence in communicating with families about unintended events. They also believed that the simulation training improved their performance by preparing them for shoulder dystocia and other perinatal emergencies (Santos, Ritter, Hefele, et al., 2015).

Shoulder dystocia management and bundle implementation. Findings related to the implementation of the new shoulder dystocia management plan and application of the shoulder dystocia bundle included the following:

• After adoption of the bundle, 99% of deliveries involving shoulder dystocia had a head-to-body delivery time of within the required 3-minute window, a finding sustained for 2 years after implementation (Cusick, Cox, Welch & Firneno, 2015).
• Reporting of shoulder dystocia increased threefold in the year following implementation, reaching 99% compliance. In the second and third year following implementation, documentation of head-to-shoulder delivery time reached 99% compliance (Burstein, Zalenski, Edward, et al., 2015).
• The grantee noted facilitators that optimized adoption of the bundle, including: practices based on evidence and guidelines, clinicians participated in protocol development and opinion leaders at each site supported it, guidelines were simple and did not require a lot of time, and clinicians could gain experience using the guidelines in a simulation before implementing it in a real situation (Burstein, Zalenski, Edward, et al., 2015).

Shoulder dystocia risk factors. The grantee enrolled about 20,000 women across the five demonstration hospitals. Generally, the findings indicate a higher risk of shoulder dystocia for non-Hispanic Black and Hispanic women, confirm well-documented risk factors (e.g., maternal age, body mass index), and suggest some newly identified risk factors (Hendrich, McKoy, Gale, et al, 2013). The specific findings of this substudy are pending publication.

Shoulder dystocia-related claims. Three years after establishing the new guidelines, none of the five demonstration sites had any malpractice claims based on shoulder dystocia (Cusick, Cox, Welch & Firneno, 2015). As a result of these safety improvements, money that had been saved for malpractice claims were diverted to support implementation of the model at 42 additional Ascension Health sites (Santos, Ritter, Hefele, et al., 2015).

Event reporting.  Following 27 months of implementation, the sites demonstrated improvement in event reporting, nearly doubling their rate of event reporting per 1,000 births (43 vs. 84 per 1,000 births; p<.01) (Santos, Ritter, Hefele, et al., 2015).

Harm. Following 27 months of implementation, the grantee also demonstrated improvement in high-risk malpractice events, decreasing the rate of high-risk malpractice events per 1,000 births by half (14 vs. 7 per 1,000 births; p<.01). Further, incidents of shoulder dystocia and fetal distress decreased 50% (Santos, Ritter, Hefele, et al., 2015).

Selected Publications and Presentations Developed by Grantee

Hendrich A, McCoy CK, Gale J, et al. Ascension Health’s demonstration of full disclosure protocol for unexpected events during labor and delivery shows promise. Health Aff. 2014 Jan;33(1):39-45. PMID: 24395933.
Santos P, Ritter GA, Hefele JL, et al. Decreasing intrapartum malpractice: Targeting the most injurious neonatal adverse events. J Healthc Risk Manag. 2015 Apr;34(4):20-27. PMID: 25891287.

Selected Other Products Developed by Grantee

• Perinatal Safety Implementation Kit containing: shoulder dystocia bundle and a sample minutes/seconds timer, Simulation and Team Training Trainer Guide, a pamphlet listing the Elearning education programs for Advanced Fetal Assessment and Monitoring, Introduction to EFM, Managing Shoulder Dystocia and Simulation and Team Training, Essentials to TeamSTEPPS^® and associated material and Early Elective Delivery, Induction and Augmentation of Labor bundles. 
• A video on high reliability and the effects of disclosure, with highlights from parents of an infant injured from birth trauma.
• Curriculum for the disclosure process.
• Caregiver survey tool to document perceptions and changes over time following disclosure and related educational offerings.

Source Documents for This Grantee Profile

Burstein PD, Zalenski DM, Edward JL, et al. Changing labor and delivery practice to achieve sustainable decreases in neonate injury. Manuscript submitted for publication. 2015.
Cusick W, Cox RL, Welch RA & Firneno C.  Devising obstetric best practice guidelines to address variation, improve patient outcomes, and decrease malpractice risk. Manuscript submitted for publication. 2015.
Hendrich A, McKoy C, Gale J, et al. Healing Without Harm: A Multi-Site Demonstration Project. Final Progress Report. Prepared by Ascension Health under grant number R18 HS19608. Prepared for the Agency for Healthcare Research and Quality. St. Louis, MO; Jun 2013.
Resar R, Pronovost P, Haraden C, et al. Using a bundle approach to improve ventilator care processes and reduce ventilator-associated pneumonia. Jt Comm J Qual Patient Saf. 2005;31(5):243-248. PMID: 15960014.
Santos P, Ritter GA, Hefele JL, et al. Decreasing intrapartum malpractice: Targeting the most injurious neonatal adverse events. J Healthc Risk Manag. 2015 Apr;34(4):20-27. PMID: 25891287.

Fairview Health Services

Title: Eliminating Preventable Perinatal Injuries and Reducing Malpractice Claim
Award Number: R18 HS19587-01
Principal Investigator: Kristi K. Miller, RN, MS 
Award: $2,982,690
Period of Performance: 7/01/10–12/31/13

Goals

The project, a continuation of the Premier Patient Safety Initiative (PPSI), primarily aimed to create high-reliability teams and care processes in the perinatal units of 14 acute care hospitals and, by improving safety practices, reduce harms and associated malpractice claims and costs.

Methods

This grant supported Phase II of the PPSI study (2011–2012), which built on the successes of PPSI Phase I (2008–2010). The 16 hospitals in Phase I were diverse in birth volume, academic status (teaching vs. nonteaching), size, and geography. Fourteen of the 16 Phase I hospitals continued participation into Phase II. The grantee described the project as a quality improvement collaborative, whereby interdisciplinary teams from multiple organizations learned best practices, shared experiences, and used quality improvement methods and techniques.

Phase I included the following components:

• Onsite High Reliability Perinatal Safety Assessments, which measured the safety and reliability of each hospital’s perinatal care processes
• Implementation of three standardized care processes called clinical care bundlesⁱ: elective induction, augmentation, and vacuum extraction (described below)
• Electronic fetal monitoring interpretation training for doctors and nurses using National Institute of Child Health and Human Development terminology
• Performance feedback via monthly educational webinars as well as conference calls and emails, with topics including processes, outcomes, barriers, and best practices
• Two all-team meetings to share best practices
• Periodic individual team coaching

Researchers concluded that the period of Phase I was insufficient to achieve the goal of 90 percent bundle compliance and to measure the effect of improved reliability of perinatal care on patient harm. Phase II continued Phase I components but enhanced the intervention with onsite in situ simulation and TeamSTEPPS^® training. (TeamSTEPPS^®—Team Strategies and Tools to Enhance Performance and Patient Safety—is an evidence-based teamwork system developed by AHRQ and the Department of Defense’s Patient Safety Program to improve communication among health care providers.) Further, Phase II included an evaluation of perinatal liability claim outcomes and a second round of onsite High Reliability Perinatal Safety Assessments.

Each of the intervention hospitals initiated use of the three evidence-based bundles during Phase I. Although the three bundles contain different elements, they “share a common objective of standardizing processes and reducing practice variation” (Riley, Begun, Meredith, et al., 2015, p. 8). The bundles included the following:

• Elective induction (elements include gestational age greater than 39 weeks, normal fetal status prior to start of oxytocin, pelvic exam prior to start of oxytocin, and recognition and management of tachysystole)
• Augmentation (elements include documentation of estimated fetal weight, normal fetal status, pelvic exam prior to the start of oxytocin, and recognition and management of tachysystole)
• Vacuum extraction (elements include alternative labor strategies considered, prepared patient, maximum application time and number of pop-offs predetermined and documented, cesarean and resuscitation teams available at delivery, and high probability of success)

The primary intervention introduced in Phase II was interdisciplinary team training using in situ simulation and a condensed TeamSTEPPS^® curriculum. At each site, expert clinicians conducted a 3-day training to train participants to become expert team leaders. A train-the-trainer experiential learning method was used with the goal that 100 percent of members within a multidisciplinary team (staff from labor and delivery, neonatal, operating room, anesthesia, lab, and other staff) would be trained using the in situ simulation within 1 year of the onsite training. The 3-day training included 4 hours of education on “Improving Individual and Team Communication” and “Debriefing” as well as the simulation scenarios that involved postpartum hemorrhage, uterine rupture, abruption, shoulder dystocia, and/or resuscitation of the hypovolemic newborn. Each simulation involved a setting of the stage, the simulation experience, and a facilitated debriefing that included a review of a videotape of the simulation. Performing simulation in the hospital units where clinicians work was beneficial because it revealed latent conditions related to care and allowed teams to simultaneously address processes and equipment issues. The goal of the simulations was to normalize a “culture of safety,” standardize communication, and achieve high reliability of patient care.

Across the 14 hospitals, 1,883 (45.7%) perinatal staff participated in the in situ simulation training. Sending trainers to the 14 hospitals to conduct simulations at no cost to the sites increased the sites’ commitment to the project and gave them an opportunity for experiential learning. The sites reported some challenges to participation in simulation training, including competing tasks (e.g., scheduling conflicts) and problems with physician buy-in.

Analysis

A prospective cohort study design was used consisting of a 5-year intervention period (January 1, 2008–December 31, 2012) across Phases I and II and a 2-year baseline period (January 1, 2006–December 31, 2007). For some outcomes, the study used eight geographically diverse comparison hospitals that included those with small, medium, and large birth volumes and teaching and nonteaching hospitals. The study examined the following outcomes:

• Bundle compliance. Data were collected from chart audits to assess bundle compliance.
• Safety and reliability of perinatal care processes. A 2-day High Reliability Perinatal Safety Assessment on labor and delivery care was conducted at baseline and 2012, and the AHRQ Hospital Survey on Patient Safety Culture was administered in 2008, 2010, and 2012.
• Harm and clinical measures. Hospitals submitted data on perinatal discharges to the National Perinatal Information Center (NPIC) to assess changes in the following harm measures: Adverse Outcomes Index (AOI), which divides the number of mothers or infants with an adverse event by the number of deliveries; Weighted Adverse Outcomes Scale (WAOS), which sums the weights for each of the adverse events that occurred and divides by the number of total deliveries; Severity Index (SI), which sums the weights for each adverse event that occurred and divides by the number of patients that had an adverse event; AHRQ Patient Safety Indicators (PSI) 17–birth trauma, a measure of a potential adverse event or complication experienced by a patient that could be prevented by system changes at the provider or organizational level; and other clinical outcomes (e.g., cesarean section rates). For the eight comparison hospitals used in the study, data on harm and clinical outcomes were gathered.
• Claims. The grantee accessed litigation records and malpractice claims data for the baseline period prior to implementation (January 1, 2006–December 31, 2007) and the implementation period (January 1, 2008–December 31, 2012). (Litigation records were obtained through American Excess Insurance Exchange through a special arrangement for this study. Malpractice claims data were available through confidentiality contracts with Premier Insurance Management Services, Inc.) For the purpose of the analysis, only records from 2008 and 2009 were used for the intervention period to account for a possible lag time in the filing and adjudication of claims. Although 14 hospitals participated, the analysis used data from only 13 in the liability assessment because of unreliable data. Researchers compared claims data two ways: (1) obstetric claims during the baseline period compared with the intervention period (2006–2007 vs. 2008–2009) and (2) the baseline-to-intervention change in obstetric claims compared with the baseline-to-intervention change in non-obstetric claims in the same hospitals.

Findings

Training. Team communication training was considered to be an important aspect of implementation, however, participation in training varied greatly by hospital, ranging from an estimated 10 percent to 93 percent of clinicians, with a median attendance of 50 percent of clinicians per participating hospital (Miller, Riley, Meredith, et al., 2013).

Bundle compliance. The study revealed increases in bundle compliance from rates established during Phase I. In Phase II, between Quarter 3 of 2010 and Quarter 4 of 2012, average compliance increased from 54 percent to 70 percent for the vacuum extraction bundle, 74 percent to 86 percent for the augmentation bundle, and 87 percent to 88 percent for the elective induction bundle. Participating hospitals improved overall bundle compliance from 72 percent to 81 percent during this time. The grantee had aimed to reach 100 percent compliance, which was achieved by several teams with specific bundles for one or more quarters; however, none of the hospitals achieved 100 percent compliance across all three bundles for any quarter. Some challenges noted to implementing the bundles included competing tasks (e.g., implementing electronic medical records), staff turnover, and a lack of active physician champions who were able to change peer behavior, and the difficulty of successfully implementing all the components of a bundle (since compliance for the bundle is scored on an all-or-nothing basis). The grantee noted that the development and use of checklists appeared to improve bundle compliance.

Safety and reliability of perinatal care processes. Overall, all 14 hospitals improved their High Reliability Perinatal Safety Assessment scores significantly from 2008 (baseline) to 2012. At baseline, 66 percent of the demonstration hospitals were partially compliant with industry standards. By the end of the project, 89 percent of the demonstration hospitals were meeting industry standards. For the AHRQ Hospital Survey on Patient Safety Culture, researchers detected significant improvement from 2008 to 2010 on only one of the 12 dimensions (staffing), and no significant improvement from 2010 to 2012.

Harm and clinical measures. During Phase II, AOI improved for implementation hospitals and worsened for comparison hospitals (p=.025), with participating hospitals outperforming comparison hospitals on rates of total adverse events and maternal adverse events. Differences between the intervention and comparison hospitals were not detected with the other measures of harm or clinical outcomes. For the intervention hospitals, from baseline through Phase II (Quarter 1 of 2006 through Quarter 4 of 2012), AOI decreased 14.4 percent (p=.032), while no significant change was detected for WAOS, SI, or PSI 17. The overall process behavior for the AOI over the 7 years includes two occurrences of special cause: (1) from Quarter 1 of 2006 to Quarter 1 of 2008 (p<.001), which corresponds to the baseline period, and (2) from Quarter 1 of 2010 to Quarter 4 of 2012, the end of the project (p<.003). Common cause variation is observed between the two process shifts.

Claims. From 2006 to 2009, the 13 hospitals had 185,373 deliveries, with 125 resulting in a lawsuit claim and 25 resulting in claims paid. Although the 25 claims represented only 9 percent of the hospitals’ share of total claims, they represented 24 percent of the amount of losses paid and 27 percent of legal defense costs. In comparing the baseline to intervention period, the grantee found a significant reduction in total amount of losses paid per 1,000 deliveries and total indemnity per 1,000 deliveries (p<.05 for both analyses). Further, in the analysis comparing claims activity during this period in obstetrics vs. non-obstetrics in the same hospitals, claims paid, total losses paid, and total indemnity paid were reduced significantly for obstetrics (43.9%, 77.6%, and 84.6%, respectively) compared with non-obstetrics (p<.05 for all analyses).

Selected Publications and Presentations Developed by Grantee

Damon AL, Parrotta CD, Wallace LA, et al. The effectiveness of providing evidence-based perinatal practice to low-income populations providing perinatal care: Does patient income influence the delivery of quality care? J Hosp Adm. 2013;2(4):82-90.
Davis SE, Miller KK, Riley W. Improving individual and team performance: The power of in situ simulation and TeamSTEPPS^®. International Meeting on Simulation in Healthcare; 2012 Jan 27-Feb 1; San Diego, CA.
Davis SE, Miller KK, Riley W. Improving individual and team performance: The power of in situ simulation and TeamSTEPPS^®. TeamSTEPPS^® National Conference; 2011 Jun 21-23; Denver, CO.
Doyle JL, Kenny TH, von Gruenigen VE, et al. Implementing an induction scheduling procedure and consent form to improve quality of care. J Obstet Gynecol Neonatal Nurs. 2012;41:462-473. PMID: 22697170.
Hansen SS, Arafeh J. Implementing and sustaining in situ drills to improve multidisciplinary health care training. J Obstet Gynecol Neonatal Nurs. 2012;41:559-571. PMID: 22548312.
Kozhimannil KB, Sommerness SA, Rauk P, et al. A perinatal care quality and safety initiative: Are there financial rewards for improved quality? Jt Comm J Qual Patient Saf. 2013;39(8):339-348. PMID: 23991507.
Parrotta C, Riley W, Meredith L. Utilizing leadership to achieve high reliability in the delivery of perinatal care. J Healthc Leaders. 2012;4:157-163.
Premier Healthcare Alliance. Reducing preventable birth injuries and liability claims through evidence-based care, enhanced teamwork. Premier Perinatal Safety Initiative, Phase I Summary, 2008-2010. 2012, December. Accessed at http://gshrm.org/images/meeting/091313/premier_white_paper_nov2012_final.pdf
Premier Healthcare Alliance. Reducing Preventable Birth Injuries and Liability Claims Through Evidence-Based Care, Enhanced Teamwork. Premier Perinatal Safety Initiative Results Summary 2008-2013. Released May, 2013. Accessed at http://aeixrrg.com/files/PPSI%20White%20Paper5-13-14Final.pdf
Riley W. Creating high reliability to reduce patient harm. Florida State University College of Medicine, Grand Rounds; 2012 March 1; Tallahassee, FL.
Riley W, Davis S, Miller K, et al. An analysis of latent conditions and their association with active failures through in-situ simulation [abstract]. In: Proceedings of the 11th International Meeting on Simulation in Healthcare; 2011 Jan 21-16; Phoenix, AZ. Minneapolis, MN: Society for Simulation in Healthcare; 2011:81.
Riley WJ, Meredith L. Implementing standardized clinical best practice and measuring success. Oral poster presentation. Annual Global Healthcare Conference; 2012 August 27-28; Singapore.
Riley WJ, Meredith L. Utilizing a standardized clinical best practice and assessing reliability of perinatal care. GSTF Digital Library. 2012 Dec.
Riley WJ, Meredith L, Parrotta C. The determinants of implementing best practices for perinatal units. Poster presentation. Annual Research Meeting, AcademyHealth; 2012 June 24-26; Orlando, FL.
Riley WJ, Meredith L, Parrotta C. Measuring high reliability in hospital perinatal units. GSTF Digital Library, Journal of BioSciences. 2013 Feb.

Selected Other Products Developed by Grantee

• High Reliability Perinatal Safety Assessment
• Sets of guidelines and protocols for simulation and bundle compliance
• Simulation training tools, including a 15-minute didactic video, Improving Individual & Team Performance: The Power of TeamSTEPPS^® and In Situ Simulation
• Webinars on various topics, including bundle compliance, safety culture and accountability, physician engagement and leadership, obstetric staffing, and project updates

Source Documents for This Grantee Profile

Miller K, Riley W, Meredith L, et al. Eliminating Preventable Perinatal Injuries and Reducing Malpractice Claims. Final Progress Report. Prepared by Fairview Health Services under grant number R18 HS019587. Prepared for the Agency for Healthcare Research and Quality. Minneapolis, MN; Dec 2013.
Resar R, Pronovost P, Haraden C, et al. Using a bundle approach to improve ventilator care processes and reduce ventilator-associated pneumonia. Jt Comm J Qual Patient Saf. 2005;31(5):243-248.
Riley W, Begun JW, Meredith L, et al. Integrated approach to reduce perinatal adverse events: Standardized processes, teamwork training, and performance feedback. Manuscript submitted for publication. 2015.

Massachusetts State Department of Public Health

Title: Medical Malpractice & Patient Safety Proposal
Award Number: R18 HS19508-01
Principal Investigator: Madeleine Biondolillo, M.D.   
Award: $2,912,566
Period of Performance: 7/01/10–3/31/14

Goals

The Proactive Reduction of Outpatient Malpractice: Improving Safety, Efficiency, and Satisfaction (PROMISES) project aimed to determine whether implementation of one or more quality improvement (QI) activities in a group of Massachusetts outpatient primary care practices would improve safety and reduce medical malpractice risk in targeted “3+1” risk areas that contribute to a large proportion of medical errors in these settings (laboratory test results management, referral management, and medication management, as well as communication among care providers and with patients). The project extended the work of the Harvard Malpractice Study by broadening the focus from hospitals to the ambulatory realm where most health care is delivered.

Methods

This grantee recruited 25 small- to medium-sized adult primary care practices in Massachusetts to participate in this randomized control trial. Sixteen of the practices were randomly assigned to participate in a 15-month intervention (from January 2012–April 2013). The grant, led by the Massachusetts Department of Public Health, involved collaborators in the State’s public and private sector, many of which had a track record of successful collaborations: Massachusetts Coalition for the Prevention of Medical Errors, a public–private partnership comprising health care stakeholders including consumer organizations; the Institute for Healthcare Improvement, offering QI expertise; Brigham and Women’s Hospital, Center for Patient Safety Research and Practice, offering clinical and research expertise; two insurance companies, CRICO/RMF and Coverys, who together covered a large majority of the State’s physicians; Harvard Schools of Medicine and Public Health, providing expertise in organizational behavior and statistics; and Health Care for All, a consumer group knowledgeable about patient engagement.

The pre-existing relationships and “the degree of trust” among the various PROMISES collaborators were mentioned as a facilitator to getting buy-in to the project from both insurance companies. The partnership of collaborators was described as “strengthening” over time, with greater “collegiality.” For this group and its subgroups, a profound commitment to the cause was the driving force.

The interventions implemented for 15 months in the 16 intervention practices targeted the 3+1 key risk areas, with each of the 16 practices choosing 1 or more of the 4 areas to improve: laboratory test results management (11), referral management (11), and medication management (3), as well as communication among care providers and with patients (13). Some targeted areas of improvement specific to communication included communication with the patient before, during, and after an office visit and methods for engaging patients who have concerns or insights about their care.

The grantee established a quality improvement learning collaborative with the 16 practices, which participated to varying degrees in monthly didactic and interactive webinars taught by workflow and efficiency experts and quarterly face-to-face learning sessions; they also received ongoing coaching by an Improvement Advisor with expertise in quality, efficiency, and safety. The practices implemented the Model for Improvement, including the Plan, Do, Study, Act method. They “were coached to perform rapid, small-scale tests of change and to iteratively improve performance of high-risk clinical systems, as well as to embed simple measurements into routine work streams to guide improvement efforts” (Schiff, Reyes Nieva, Griswold, 2015, p. 4). The adjustments were intended to improve inefficient and ineffective processes that could affect patient safety liability. The Improvement Advisors had weekly communication with the intervention practices by email and phone and visited the sites once or twice monthly for onsite coaching. The Improvement Advisors determined where each site was on the “innovation curve” and responded to their specific needs. Based on the variation in the sites’ innovation readiness, support needed to be targeted to each practice.

Each site selected three or more people to serve on its QI team; members were usually a practice manager, a physician, and one or more members of the nursing or administrative staff. The grantee encouraged the sites to select team members to ensure representation of a senior leader, a clinical champion, and a day-to-day champion.

Through the intervention, the practices aimed to show improvement in one or more of the following four key system improvement drivers: (1) culture of quality and safety, (2) effective communication and collaboration, (3) reliable tracking and management processes, and (4) enhanced operational efficiency.

Challenges to implementation included short timeframes for start-up, delays in obtaining institutional review board (IRB) approval, resistance to change among physicians and office staff, multiple and competing priorities (e.g., meaningful use, other QI projects), staffing issues (e.g., chronic understaffing, turnover in leadership positions), and local IT workflow issues.

Analysis

This cluster randomized control trial collected data through chart reviews and surveys. In addition, a retrospective claims analysis was conducted before implementation.

Claims. Prior to implementation, the grantee’s partners CRICO/RMF and Coverys analyzed closed claims data for a 5-year period (January 1, 2005–December 31, 2009) to assess key failures contributing to medical errors and medical claims in primary care. Claims data were provided by CRICO/RMF and Coverys. All claims involving primary care practices insured by CRICO/RMF and Coverys were screened.

Chart reviews. Chart reviews were conducted by the grantee at each of the intervention sites before and after the intervention to capture objective information about how abnormal lab results, referrals, and medications were handled; up to 100 charts were reviewed at each of the 16 sites. Chart reviews were triggered by a set of defined abnormal test results. Specifically, “reviewers examined charts to determine if a) abnormal test results were present in the chart, b) the abnormality was noted by a responsible provider, c) there was documentation of an action or referral plan, d) the patient was notified of the abnormality, and e) there was evidence that the treatment or plan was completed for these trigger results as well as other predefined high-risk results or findings” (Schiff, Reyes Nieva, Griswold, 2015, p. 5). In total, the grantee reviewed 815 charts before the intervention and 762 charts afterward, representing 1,629 and 1,530 abnormal lab tests, respectively.

Surveys. Improvement Advisors were asked to state which intervention sites were “more likely” or “less likely” to show an impact based on their experience working with the sites (i.e., the site’s level of engagement).

Staff (e.g., providers, practice managers, administrators) at intervention and control sites were administered a 63-item, online survey both before and after the intervention. The survey assessed 11 domains related to the targeted areas and other areas (e.g., malpractice concerns, patient-focused care, teamwork). Across all 25 sites, 292 and 287 staff completed the pre- and posttest surveys, with a response rate of 61 percent and 60 percent, respectively. In addition, administrators in the 25 practices were given an online survey before and after the intervention that assessed the existence of standardized safety processes in the four targeted areas and the practice’s technology capabilities. The response rates for administrators were 61 percent and 100 percent at baseline and post-intervention, respectively. Staff from the 16 intervention practices additionally completed a practice characteristics questionnaire and participated in site visits, interviews, and observations of their workflow and processes.

Patients at intervention and control sites were surveyed both before and after the intervention using a 34-item paper-and-pencil survey based primarily on the CAHPS Patient-Centered Medical Home Survey as well as other instruments. The survey covered seven domains related to patient experience (e.g., level of trust, communication, patient-centered care). A group of 150 patients from each site were randomly selected to receive the survey, which was sent by mail. Across all 25 sites, 1,767 and 1,521 patients completed the pre- and posttest surveys, with a response rate of 48 percent and 42 percent, respectively. In terms of the process of administering the patient survey, collecting patient names for the survey was a major undertaking for some practices.

Exit interviews. Exit interviews were conducted with leaders at all intervention sites.

Findings

Claims. In the retrospective analysis of closed claims data, 551 closed claims from the primary care setting represented 7.7% of all closed claims (7,224). These 551 claims named 595 physicians, with some physicians being named in two cases (53) or more cases (17). Most (72%) of the 551 cases were related to failures in the diagnostic process, most frequently for some form of cancer. In terms of the outcomes of claims, the grantee found that claims in primary care, especially those involving diagnostic error, were significantly more likely than claims in other medical settings to be settled (14% vs. 10%) or result in a verdict for the plaintiff (35% vs. 20%) (p<.001).

Safety practices. From pre- to post intervention, chart reviews revealed significant improvements in four of the five measures of abnormal test follow-up: result not in chart (2.2% vs. 0.8%; p<.001), action plan not documented (20.4% vs. 14.4%; p<.001), patient not notified (20.8% vs. 15.0%; p<.001), and action plan not completed (19.3% vs. 10.8%; p<.001). Chart reviews from pre- to post intervention showed improvement in the rates of potential risks: per 1,000 patients with an abnormal lab result, rates of potential safety risks decreased from 155 to 54 and rates of serious potential safety risk decreased from 28 to 13 (p<.05).

Patient and staff perceptions and attitudes. Survey responses from patients before and after the intervention were positive or very positive across most domains. At pretest, patients provided more than 500 comments in response to an item requesting suggestions for the doctor’s office to improve the care and services it provides. The Improvement Advisors shared the results with providers at their respective sites, who appreciated the suggestions and took them into consideration when planning improvement activities. No significant pre- to post intervention improvement in the intervention sites compared with the control sites was found. However, among intervention sites, those identified by Improvement Advisors as more engaged had significant patient-rated improvement in some domains than those identified as less engaged.

Survey responses from clinic providers and staff before the intervention revealed a lack of fully implemented safety systems and processes in at least one-third of practices, which—in combination with communication and safety culture problems—“frequently left staff feeling vulnerable to malpractice suits.” Respondents felt the least positive in the areas of referral management safety, talking openly about safety problems, willingness to report mistakes, and feeling rushed while caring for patients.

From pre- to post intervention, intervention sites had greater improvement on mean staff ratings than control sites overall and in the three high-risk domains targeted by the intervention; however, these differences were not significant. Based on mean scores, intervention sites improved relative to control sites on 9 of the 11 domains, with no significant differences. In an analysis of the percent of negative responses, intervention sites improved relative to control sites on 7 of 11 domains, with significant differences in test results management (p<.001) and teamwork (p<.05). Few significant differences were detected across the domains between intervention sites identified by Improvement Advisors as more engaged versus less engaged.

In exit interviews, primary care practices provided positive feedback on PROMISES, particularly the individualized support from coaches, participation in the learning collaborative, and use of the Model for Improvement and PDSA cycles.

Selected Publications and Presentations Developed by Grantee

Kerrissey M, Satterstrom P, Leydon N, et al. Integrating vs. isolating: The integrative role of clinic managers in improving primary care. Manuscript in preparation. 2015.
Reyes Nieva H, Foskett C, O’Malley M, et al. Baseline malpractice and safety risks in 3 key areas: Failures in high-risk outpatient test, referral, and medication management and documentation. Manuscript in preparation. 2015.
Schiff G, Griswold P, Ellis BR, et al. Doing right by our patients when things go wrong in the ambulatory setting. Jt Comm J Qual Patient Saf. 2014;40(2):91-96. PMID: 24716332.
Schiff G, Puopolo AL, Huben-Kearney A, et al. Primary care closed claims experience of Massachusetts malpractice insurers. JAMA Internal Medicine. 2013;173(22):2063-2068. PMID:24081145.
Singer SJ, Reyes Nieva H, Brede N, et al. Evaluating ambulatory practice safety: the perspectives of administrators and practice staff.  Med Care. 2015; 53(2):141-152. PMID: 25464161.

Selected Other Products Developed by Grantee

• When Things Go Wrong in the Ambulatory Setting, a 4-page tool published in 2013 that is a companion to When Things Go Wrong: Responding to Adverse Events (2006)
• When Things Go Wrong in the Ambulatory Setting video, available at https://vimeo.com/76550944
• PROMISES Patient Safety Curriculum, 14 Web-based modules and materials (e.g., videos, handouts, tools) to improve patient safety in the ambulatory setting
• Continuing medical education (CME) materials for sessions that involve practice coaches and an interpractice collaborative learning network (CME credit was available through the Massachusetts Medical Society Web site, http://www.massmed.org/Continuing-Education-and-Events/Online-CME/Online-CME-Courses/#.U6olH_ldVvU)
• Evaluation tools, including patient survey, staff and provider survey, administrator survey, chart review instrument, qualitative exit interviews instrument, and analysis of practice change management
• Malpractice closed claims database
• Automated, abnormal lab query reports in collaboration with Quest Diagnostics, Inc., for assistance with PROMISES postintervention chart review.
• PROMISES project Web site, www.brighamandwomens.org/pbrn/promises, that includes many of the project’s products

Source Documents for This Grantee Profile

Biondolillo M, Schiff GD. PROMISES Project. Final Progress Report. Prepared by Massachusetts State Department of Public Health under grant number R18 HS019508. Prepared for the Agency for Healthcare Research and Quality. Boston, MA; Mar 2014.
Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Lawthers AG et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med 1991; 324(6):370-376.
Schiff G, Reyes Nieva H, Griswold P, et al. Reducing ambulatory safety risk: Results of Massachusetts PROMISES project. Manuscript in preparation. 2015.

---