Improving the Measurement of Surgical Site Infection Risk Stratification/Outcome Detection

A Users Guide for "Improving the Measurement of Surgical Site Infection (SSI) Risk Stratification and Outcome Detection"

October, 2011

Prepared by:
Jef Huntington
Josh Durfee
Connie Price
Lucy Savitz

Supported by ACTION Contract No. 290-2006-00-20, TO No. 8

Infection Prevention Support Tool
A Users Guide for “Improving the Measurement of Surgical Site Infection (SSI) Risk Stratification and Outcome Detection

Surveillance is an integral part of the day-to-day duties of an Infection Preventionist (IP). Chart reviews conducted for the detection of surgical site infections (SSIs) can be cumbersome. Methods to improve the efficiency of manual chart abstractions are welcomed by overburdened IP clinicians. Towards that end, the Agency for Healthcare Research and Quality (AHRQ) sponsored a project carried out by investigators from Denver Health, Intermountain Healthcare, and the Salt Lake City Veterans Affairs Medical Center (VAMC)¹.

In an effort to enhance detection and measurement of SSIs, the AHRQ funded study developed and tested a computer-assisted, IP algorithm for SSI surveillance with a focus on coronary artery bypass graft or CABG, herniorrhaphy, and total hip and knee arthroplasty procedures.

The purpose of the IP algorithm is to flag charts that are not likely to contain an SSI. By removing charts without potential SSIs, the IP's work-load is reduced and the likelihood of effectively and efficiently identifying an SSI using traditional manual review is increased.

Purpose of the SSI Algorithm and Accompanying Chart Review

1. Running the algorithm removes charts that are unlikely to contain an SSI and improve an IPs ability to identify a true infection.
2. Reducing the amount of time IPs spend doing chart reviews searching for instances of SSIs frees them to dedicate more time for prevention activities.

SSI Algorithm

The SSI-identifying algorithm is a program written with structured query language (SQL) that uses Boolean logic (in a “case” statements) to identify potential surgical site infections. Setting up the query is a one time, up-front investment that results in a generalized program that can be reused repeatedly. The time required to set up the algorithm varies based on the electronic availability of required data fields. If all the variables are easily accessible, the algorithm can be programmed in 1-2 hours. The most difficult aspect of developing the query is the complexity inherent in joining multiple data sources such as laboratory, surgery, and patient demographic data.

The program was originally written with the intent of simultaneously flagging SSIs for each of four different procedure types: CABG, herniorrhaphy, and total hip and knee arthroplasty. However, the query can easily be modified to independently evaluate one, two, three, or four procedures. Further, the program can be modified to address other surgical procedures. Below we provide the code that can be modified to suit your organization's specific needs.

Explanation of Algorithm Program Variables

The total_ssi variable is optional and is derived from the results of previous chart reviews. The purpose of this variable is to identify false positives and false negatives and calculate an odds ratio.; The purpose of total_ssi is for comparison with algorithm_ssi.

Using Algorithm Query Results

Once the algorithm has been executed, the results should be downloaded to a spreadsheet so that a two-by-two table can be created to identify false positives, false negatives, and an odds ratio². Within a spreadsheet software program such as Microsoft Excel, a pivot table can be used to create the two-by-two table of ‘total_ssi' vs. the ‘algorithm_ssi'.

The spreadsheet will contain the following variables:

1. EMPI – unique patient identifier
2. Postopabx: This column indicates whether they received antibiotics 3 or more days postoperatively (0/1)
3. Postopcx – A post operation microbiology culture 3 or more days post surgery (0/1)
4. Laboratory Values: Your site may have collected additional laboratory values on surgical patients such as ESR, WBC, CRP, NE_P (Neutrophil percentage). You will not need to do anything with this information
5. Postopadmit – This column will tell you if the patient was admitted after the surgery
6. Sgytype – Surgery type (‘CABG', ‘HERNIA', ‘NEE', and ‘HIP')
7. Total_ssi: this column shows the results of Infection Preventionist chart reviews at your institution. A 1 represents a positively identified SSI from previous surveillance. A 0 represents the absence of an SSI from previous surveillance. ICP did not identify this surgical case as an SSI during previous surveillance
8. Algorithm_ssi: this column indicates whether the algorithm codes identified the surgical case as having an infection. A 1 denotes a positive SSI and 0 denotes the absence of an SSI
9. Birth Date: identifies patient's date of birth (not specified in the query, but needed for the Access data entry tool)
10. Surgery Date: date of surgery for review purposes (not specified in the query, but needed for the Access data entry tool)

It is recommended that a system-specific test be conducted at your organization to ensure the reliability of results. The suggested chart review process is provided in the following section.

Chart Review Process

After running the algorithm, it may be beneficial for an IP to perform chart reviews on the false positive and false negative records. The Salt Lake City VAMC investigators developed a Microsoft Access data entry application that can be used to capture chart review results from the false positive and false negative records³. The Access application is not mandatory for the chart reviews; however, it is helpful for organizing the results and capturing pertinent information. The Access database has two features, an uploading feature and a chart abstraction feature. To upload the data, a Microsoft Excel spreadsheet is created with the following variables (variable names must match exactly):

• PatientIdentifier
• DateOfBirth
• DateOfSurgery
• SurgeryType

Uploading the spreadsheet is simple. Double click on the “LoadAnnotator: Table” option in the left pane, then import the Excel spreadsheet. Once the spreadsheet is uploaded, the data must be loaded in to the annotation tool. To do this, click on the “qryInsertSSILoad” option in the left pane. If Access gives a security warning about a “VBA Macro,” simply select “Enable this content” then click on “OK.” Several new menus will appear, select “OK” on both and the data will be loaded into the annotate table.

1. Using the MRN found in the Excel spreadsheet, you will look up the patient in your medical record system to review the case.
2. Locate all inpatient and outpatient notes for the surgical date and procedure as indicated in the Excel spreadsheet.
3. Review the chart to determine whether the patient developed an SSI for this surgical procedure. Use the CDC- NHSN SSI guideline to define whether SSI is present and type (superficial, deep, organ space). You will need to define the type (superficial, deep, organ space) in the database.
4. As you review the chart, consider the components of the algorithm for the surgery you are reviewing. If the patient had an SSI, determine if there are additional elements that are not in the algorithm that you think should be included to increase the sensitivity of the algorithm.

For the original study, each site's investigator agreed to perform chart reviews on all false positive records and all false negative records unless the combined total exceeded 50 records. If the number of records was more than 50, a random selection of false positive and false negative records was to be chosen. Intermountain Healthcare, for example, only had 16 false negatives but exceeded 50 false positives. Intermountain used the SASâ SURVEYSELECT procedure to randomly select 34 false positive records so that there would be a total of 50 records to be reviewed. Microsoft Excel also has a random number generator that could have been used for the random selection of patients (the =RAND() function). Once the patients were selected, their records (i.e., the 4 variables mentioned above) were loaded into the Access data application.

The Microsoft Access application then provided a data entry screen specifically customized for each patient to assist in the review process. When performing the chart abstractions, the Access data entry tool will ask for the following information:

• Patient Identifier
• Date of Birth
• Date of Surgery
• Surgery Type
• SSI Found (Y/N)
• SSI Type
• Reason for Failure
• Missing Information
• Missing Information Location
• Time of Information
• Algorithm Fixable
• Comment
• Entry Date

Data Entry Procedure

Start by opening the SSIAnnotator.accdb with Microsoft Access (screenshots are specific to Microsoft Access 2007). Notice the security warning in the upper left corner of the screenshot. In this case, select “Enable this content” then click on “OK.”

The next step is to load the patient identifiers. This is done by loading the double clicking on LoadAnnotator : Table in the left column. A spreadsheet-like table will open. Once the new table is open, click on the External Data tab and select the Excel option.

Once your patient specific data has been uploaded, you must load that patient information into the data entry application. To do this, select the qryInsertSSILoad option in the left pane. This will bring up a dialogue box, select Yes in this box and in any other dialogue boxes that appear. Once this is done the tool is ready to be used for chart abstraction.

Once the data is loaded, select the frmSSIAnnotate option from the left pane and the data entry tool will appear with all of the patient information pre-loaded.

Once the chart reviews are complete, the results can be easily exported from Microsoft Access to a spreadsheet for summarization. At Intermountain Healthcare, an independent registered nurse not employed by Intermountain Healthcare, who familiar with Intermountain's electronic medical records, to perform the chart reviews. The nurse indicated that each chart review took between 15-30 minutes per chart review with associated data entry into the database.

Example, Tailored Application of the SSI Detection Algorithm

Denver Health (DH) sought to extend the algorithm application to additional procedure areas and to actively implement the detection tool into routine practice. Below we provide a case study exemplar of that application.

The infection control team sought to further adapt, tailor, and validate the electronic detection algorithm for use in everyday SSI surveillance at DH in order to reduce the burden of chart review while also identifying a high percentage of SSIs for detailed investigation and prevention activities.

DH's Infection Prevention Data Manager generated a retrospective cohort of procedures, including associated SSI as defined by the National Healthcare Safety Network (NHSN)⁴ definitions, using DH surveillance data from 2007-2010. Procedures included hip and knee arthroplasty, abdominal and vaginal hysterectomy, spinal fusion, craniotomy, and herniorrhaphy.

The suggested algorithm components were re-evaluated in a DH-specific context in order to determine which variables were most easily and reliably obtained through DH's centralized data warehouse. These variables were then tested for the frequency of their occurrence in a cohort of patients in the cohort, as well as in non-cohort patients. A modified loose algorithm was developed so that the presence of any component would trigger the associated patient procedure for review.

Variables included in the DH modified algorithm were leukocytosis (white blood cell count > 10,000 cells/mL), a culture (regardless of result), or a follow-up visit associated with any of a list of SSI-related ICD-9 codes (go to Table 1).

Table 1. ICD-9 codes associated with follow-up visits in DH algorithm

Results from Running the DH Modified Algorithm. DH identified 2,179 procedures for inclusion in the cohort. Sixty procedures were associated with SSI after manual chart review using NHSN methodology (go to Table 2). The modified algorithm flagged 804 procedures (or 37% of total charts) for review. The percent of total procedures flagged for review varied by procedure type, and ranged from 15% (herniorrhaphy) to 62% (craniotomy). The modified algorithm achieved 100% sensitivity and 72% specificity in detecting SSIs validated on 4 years of our manual SSI surveillance data using NHSN methodology.

Table 2. DH Modified algorithm performance data

Potential for Savings. Over the 4-year period, 1,375 unnecessary chart reviews would have been avoided without sacrificing detection of a single SSI. Assuming 20 minutes per chart for manual review, 57 full days of chart review would have been eliminated using the algorithm for surveillance of SSI in hip and knee arthroplasty, abdominal and vaginal hysterectomy, spinal fusion, craniotomy, and herniorrhaphy.

Conclusions. DH was able to successfully adapt, tailor, and validate the electronic detection algorithm to determine SSI rates for an expanded set of surgical procedures, including hip and knee arthroplasty, abdominal and vaginal hysterectomy, spinal fusion, craniotomy, and herniorrhaphy at a safety net hospital. The modified algorithm was tailored to our setting to be 100% sensitive, while still reducing overall chart review burden by 63%, which, over a 4-year period would have saved 57 full days of chart review at our institution, allowing more time for education and other active IP interventions. The successful adaptation of the algorithm for regular use by an individual who was uninvolved in its development demonstrates the potential for translation into practice on a broad scale.

Directing Questions/Requests for More Information

For more information or questions related to content included in the User Guide, contact:

Susan L. Moore, MSPH
Assistant Director, Health Services Research
Denver Health and Hospital Authority
777 Bannock St. MC 3240
Denver, CO 80204-4507

Phone: 303-602-2744
Fax: 303-602-2741
Email: susan.moore@dhha.org

¹ ACTION Contract No. 290-2006-00-20, TO No. 8 to Denver Health.
² Odds ratios are simply a ratio of odds; in general they refer to the ratio of the odds of an event occurring in the exposed group versus the unexposed group.
³ This tool is available upon request from the research team.
⁴ The National Healthcare Safety Network (NHSN) is a secure, internet-based surveillance system that integrates and expands legacy patient and healthcare personnel safety surveillance systems managed by the Division of Healthcare Quality Promotion (DHQP) at CDC.