Environmental Scan of Patient Safety Education and Training Programs

The environmental scan, as proposed in the deliverable, Methodology and Inclusion/Exclusion Criteria,¹ served as the foundation for the electronic searchable catalog and, as such, required an inclusive and methodologically rigorous approach. During the environmental scan, AIR identified patient safety programs, using publicly available sources. The purpose of this step was to identify a comprehensive set of programs that met predetermined inclusion criteria and collect similar information about each of the programs to enable a standardized presentation in an electronic catalog. The environmental scan consisted of the following four primary steps:

• Define patient safety.
• Identify sources of information.
• Determine inclusion.
• Track results.

Define Patient Safety

As a preliminary step in the refinement of the environmental scan methodology, we conducted a literature review to identify various definitions of patient safety from reputable sources, including books, scholarly journals, Federal Government agency reports, and organizational resources. Exhibit 2 provides the most relevant definitions with their associated references.

Exhibit 2. Relevant Definitions of Patient Safety

Based upon our findings and the primary objectives of this effort, we developed a meta-definition of patient safety, combining the most meaningful components of the available definitions. Through coordination with AHRQ project officers, AIR refined this meta-definition to establish the final definition below:

Patient safety is the prevention and amelioration of adverse outcomes or injuries stemming from the process of health care, as well as initiatives aimed towards improving patient safety processes and outcomes.

This definition of patient safety was used to steer all scanning activities and serves as the primary basis for inclusion in the catalog.

Return to Contents 

Identify Sources of Information

We targeted two types of information sources during the environmental scan process: (1) peer-reviewed literature; and (2) Internet and grey literature for prior, new, and existing patient safety efforts. The literature search began with defining a set of uniform keyword search terms (go to Appendix A for a list of the terms used during this search).

Peer-Reviewed Literature

Using the list of keyword terms, the team searched medical and social science peer-reviewed literature, including both descriptive qualitative and quantitative studies, using PubMed®, PsycInfo, Cumulative Index to Nursing and Allied Health Literature (CINAHL), and other databases, as shown in Exhibit 3. In addition, we scanned conference programs for relevant proceedings, such as the Institute for Healthcare Improvement and the National Patient Safety Foundation national meetings. The scope of the search was limited to patient safety education and training resources developed domestically and focusing, in whole or large part, on critical patient safety topics and issues. The searches were limited to English-language articles with abstracts published since 1999.

Exhibit 3. Databases to be Used in the Environmental Scan

The peer-reviewed literature search yielded critical information about best practices for evaluating and implementing patient safety education and training. When reviewing journal articles, we identified a program anonymity trend. That is, the peer-reviewed literature tended to focus on different approaches to patient safety education and training or evaluation of programs; rarely, however, did this body of literature name actual programs. In many cases, we extended our searches by seeking information about the authors and developing additional search terms for the environmental scan. For this reason, the main focus of the environmental scan was on the Internet and grey literature.

Internet and Grey Literature

As with peer-reviewed literature, we used the pre-identified set of uniform keyword search terms that were keyed in a variety of search engines listed in Exhibit 4 to search the Internet.

Exhibit 4. Search Engines Used in the Environmental Scan

We began the search by scanning Web-based sources to identify prior patient safety education efforts that might serve as a foundation for current initiatives (e.g., Health Resources and Services Administration-funded research initiatives). Additionally, we identified grey literature, unpublished literature, and Internet sources that describe current and existing education and training programs.

To scan the grey literature, we explored the results from a variety of search engines, including Google, Yahoo, Bing, and others to ensure that some challenges associated with Internet searches (e.g., search engine optimization and differing search algorithms) were accounted for as much as possible. It quickly became clear that some search engines only provided aggregated results from the more popular search engines (i.e., Google and Bing). For this reason, we limited the environmental scan to Google and Bing. It should be noted that although Google and Bing results yielded a high degree of overlap, we used both search engines to ensure no programs were missed.

Return to Contents 

Determine Inclusion

Once a program was identified, we then applied a set of inclusion criteria to ensure only relevant programs would be fully abstracted and documented in the final catalog. AIR, in collaboration with AHRQ, identified the following inclusion criteria.

• Is the core content of the training program truly patient safety oriented? Given the purpose of this project, all programs to be included in the catalog must have a patient safety orientation. This criterion was intended to eliminate programs that did not fall within the patient safety spectrum, such as Customer Focus Inc.'s Patient Satisfaction Skills Training Program, which focused primarily on improving patient satisfaction and Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) scores. Inclusion was determined using the meta-definition of patient safety provided earlier in this chapter.
• Is the program based on core instructional objectives? A good educational or training program should be founded on a set of core instructional objectives. Objectives can be learning, program-based, or skill-oriented and do not necessarily have to be measured or assessed. A program that does not specify any instructional objectives may be an indicator of a less structured program. An example of a program that was excluded from the catalog is the Putting Humor to Work to Improve Patient Outcomes training program offered by Creative Training Solutions because no explicit instructional objectives were provided.
• Is the target audience health care professionals, patients and families, or another stakeholder group? The program should have a clearly defined target audience. This requirement is less strict than the others, but in essence, we wanted to ensure that included programs addressed a specific health care-related target audience or multiple health care-related audiences. This criterion was intended to eliminate programs that do not have a clearly defined health care target audience, such as the National Association of Safety Professionals' Safety Manager/Training Certification Course, which does not focus on health care professionals or any of the health care stakeholder groups, but rather on safety managers in general.
• Is the education or training program currently being offered in the United States? The program must be offered or publicly available in the United States. As a general rule, programs more than 5 years old were excluded, as were programs not currently offered or available in the United States, to ensure that the resulting catalog provides information about programs available for current use by stakeholder groups. Included programs could be dormant (with no active training occurring, but could be arranged), and the source material did not have to be U.S.-based, the program just had to be available for implementation within the United States. An example of a program that was not included is the Universitair Medisch Centrum (UMC) Utrecht Patient Safety Training Program in the Netherlands, which was a 2-day course offered in 2006 but only available and accessible to medical residents at UMC Utrecht.
• Is the training program designed for another industry and merely applied to quality improvement and patient safety? Finally, the program must not have been simply applied to the health care setting but must have been specifically tailored for this setting. Training programs designed for another industry that can be applied to quality improvement and patient safety were only included if efforts were made to adapt the program to the needs and characteristics of the health care setting. The intent was to eliminate a broad spectrum of programs that could, in theory, be applied to health care but that have not been contextualized or adapted in any way, such as DuPont/Coastal's PeopleSafety Training, which has not been tailored specifically to the health care setting.

To ensure the above criteria were applied properly, researchers conducted a pilot test using five patient safety training programs. Researchers individually applied the criteria for five identified programs. Once this exercise was completed, the researchers discussed the application of the criteria and assessed the extent of inter-rater agreement. To ensure that the inclusion criteria would be applied consistently, AIR conducted a frame-of-reference training with all researchers to ensure a shared mental model of appropriate criteria application. In cases where a researcher had questions about the application of the inclusion criteria, a second researcher was asked to evaluate and discuss the inclusion criteria with respect to the particular program. In cases where consensus among researchers could not be reached, a third researcher (the Project Director or Principal Investigator) was asked to assess whether the program merited inclusion. Additionally, scanners held weekly meetings to discuss difficulty with scanning and ensure proper application of inclusion criteria.

Return to Contents 

Results of Environmental Scan

The environmental scan yielded a total of 821 potential patient safety programs. The team tended to err on the side of inclusion for programs with limited information available at the time of the scan because each program would be reviewed more thoroughly during data abstraction.

Return to Contents 

Track Results

AIR developed a Microsoft Excel worksheet to document possible patient safety education and training. Information was documented on the keyword used, the database/search engine used, the program sponsor, the program name, the Web address of the potential education/training opportunity, the link in which the program was originally identified, and a preliminary evaluation of the program against the identified inclusion criteria, as described in the next section.

Return to Contents