Throughout this project, AIR has encountered various issues that may be of interest to AHRQ. In this chapter, we highlight some of the themes resulting from each phase of the project, as well as possible next steps in maintaining and enhancing the catalog over time.
Summary of Themes
This project comprised three major steps: (1) environmental scan, (2) data abstraction and development of an electronic searchable catalog, and (3) qualitative analysis of consumer perspectives. At each point in this process, AIR identified a number of issues that influenced the resulting catalog of patient safety education and training programs, many of which have already been mentioned in this report.
Themes from the Environmental Scan
A series of themes emerged from the environmental scan, as follows:
- Peer-reviewed literature did not yield names of specific programs.
- Different search engines led to multiple links to the same programs.
- A significant number of program sponsors did not provide sufficient information, which, in some instances, made it difficult to determine what was actually a patient safety program or a hospital initiative without a core patient safety component.
- The environmental scan yielded many links to articles, documents, and programs that were either outdated or not publicly available.
Themes from Data Abstraction and Catalog Development
Themes emerging from the data abstraction phase include the following:
- Many programs were not included in the final catalog due to the brevity of the information available on the Internet.
- Many programs were ultimately excluded from the catalog when their sponsors did not respond to subsequent inquiries to learn more about their programs.
- The majority of programs included in the catalog did not specify information regarding several data fields (e.g., AHRQ Tools & Resources Used, Program Focus, and Approaches to Implementation or Recommendations for Roll-out/ Implementation).
- A number of QIOs were excluded because they were not providing training at the time of data abstraction.
Themes from Consumer Interviews
As highlighted in the previous chapter, the interviews yielded several general themes regarding consumer perspectives of patient safety education and training programs. Included in these themes are:
- The perception (or misperception) that off-the-shelf programs cannot be customized to meet organizational needs and that they are more expensive than developing or delivering programs internally.
- Programs rarely indicate whether program evaluation measures or studies had been conducted.
- Assess needs of catalog users to identify ways the catalog can better support these needs—i.e., determine the types of information users would need for optimal use of the database.
Next Steps
Based on the lessons learned throughout this project, AIR recommends that AHRQ consider some important follow-on activities at the close of this contract. Namely, we suggest that AHRQ consider how to maintain the catalog to ensure it contains current information about available patient safety programs, as well as some additional studies to improve and extend the resources AHRQ provides its constituents.
Catalog Maintenance
The final catalog consists of 333 patient safety education and training programs, currently available in the United States. It should be noted, however, that this catalog captures only a snapshot of what is available. Obviously, new programs are continually being developed, old ones retired, and others revised and improved. In order to capture the ever-changing landscape of educational and training opportunities in the patient safety realm, AIR recommends that AHRQ consider a maintenance plan for this catalog.
In particular, AHRQ should consider a plan for periodically monitoring the Internet for new programs, revisions to programs already included in the catalog, retiring programs no longer available, and adding new programs to the catalog. At a minimum, AHRQ should consider updating the catalog on an annual basis to reflect these potential changes. AIR assumes that in the event that the catalog is maintained on the PSNet, the PSNet webmaster will field questions, concerns, and consumer suggestions regarding the catalog and will, therefore, be prepared to document any issues or comments that arise. One area of possible concern may be vendors seeking explanations as to why their programs were excluded from the catalog.
Further Investigation
As we discovered through our interviews with consumers, there are many misconceptions regarding training and educational opportunities that exist for the patient safety audience. AIR recommends that AHRQ consider some of the following research studies to better identify the needs and issues of its constituency:
- Study catalog usage data to assess what streams of patient safety training are of greatest interest (this approach can serve as a proxy for interest and drive some policy decisions).
- Study reasons why users access the catalog (e.g., are they coming to it because they have had a patient safety problem in their organization?).
- Assess needs of catalog users to identify ways the catalog can better support these needs.
- Examine the way users implement a program identified in the catalog.
- Conduct usability testing of the catalog to evaluate and improve ease of use based on findings.
- Examine the costs associated with building a program internally versus the comparative costs associated with purchasing an off-the-shelf program and customizing it as necessary.
- Develop additional metrics to demonstrate program effectiveness beyond the traditional patient safety outcome measures, due to the fact that these outcomes are often low-base-rate events (i.e., because these events rarely occur, demonstrating that a program helped to reduce their occurrence even further may not be a fair measure of program effectiveness).
- Assess patient safety audiences to identify needs for training and/or other patient safety initiatives.
- Develop a Patient Safety Education Accreditation program by leveraging information obtained through the suggested studies and the elements of effective, quality patient safety programs such as the Patient Safety Improvement Corps program.
