Improving the Measurement of Surgical Site Infection Risk Stratification/Outcome Detection

Improving the Measurement of Surgical Site Infection (SSI) Risk Stratification and Outcome Detection

Research Team Meeting at Intermountain Healthcare
36 South State Street, 16th Floor, IHCDR TWIV Room
Salt Lake City, UT 84111

20 November 2009

Attendees: Lucy Savitz, Connie Price, Walt Biffle, Susan Moore, Jason Scott, Russ Staheli, Josh Durfee, Pat Nechodom, Makoto Jones, Michael Rubin, Josh Spuhl, Scott Evans, Matt Saymore

Review of project goal

• The inclusion of inpatient and outpatient procedures will be clarified in the workplan by Lucy.
• The intent to maintain the scope of the project was discussed, for example, mapping the ICD-9 procedure codes to CPT codes within the proposed workplan.

Project management/Logistical discussion

• Review of communication/support
  • Monthly reports to cover the 14th through the 13th of the months being reported.
  • Monthly calls with the CDC team on the 3rd Tuesday of every month at 12:00 pm MST.
  • The coordination of a team call prior to the call with the CDC was requested.
    • Jason will work to set-up the team calls the week before the CDC calls.
    • The team will review the project timeline and deadlines and update task progress during the calls.

Business/Logistics

• Status of Subcontracts- Susan is working to complete subcontracts with the organizations and is expecting finalization shortly.
• Invoicing is to be reported monthly. Is it possible to report quarterly?

Task Status Reports

• Task 2
  • (2.1) mapping the ICD-9 procedure codes to CPT codes- initial mapping from the SLC VAMC complete.
• Task 3
  • (3.1) Evidence based risk factors- Scott is performing literature review and compiling a list of risk factors:
    • Focus on patient factors.
    • Breakdown modifiable vs. non-modifiable factors.
    • Rank factors in order of importance.
• Task 4
  • The first focus group is scheduled at the Academic Surgical Conference in February 2010.
    • Walt is working on the planning and logistics of organizing the group.
    • Submit IRB review.

Task Order No. 8
Improving the Measurement of Surgical Site Infection (SSI)
Risk Stratification and Outcome Detection

TWIV Conference Room
Key Bank Building
36 South State Street, 16th Floor
Salt Lake City, UT 84111

15 October 2010

Team Meeting
Agenda

Task Order No. 8
Improving the Measurement of Surgical Site Infection (SSI)
Risk Stratification and Outcome Detection

36 South State Street, 16th Floor
Salt Lake City, UT 84111
15 October 2010

Team Meeting
Summary Notes

Attendees: Susan Moore, Connie Price, Josh Durfee, Heather Gilmartin, Jef Huntington, Scott Evans, Makoto Jones, Mike Rubin, Pat Nechodom, Lucy Savitz, Jason Scott, Matt Samore
Conference call participants: AHRQ TOO: Kendall Hall; CDC Technical Advisors: Teresa Horran, Jonathan Edwards

9:00am           Meeting was convened in Salt Lake City at Intermountain Healthcare Central Office

9:30-10am     Review of Project Milestones and Deliverables

• Invoicing status; emphasizing the importance of timely invoicing.
  • Intermountain has billed 25% of budgeted resources—glitch in personnel time reporting, being fixed.
  • SLC VAMC has billed 46%.
  • Vail has billed 50%.
  • Denver Health has billed 66%.
• Nursing focus group
  • Given the delay of conducting the nursing focus group as originally planned in the proposal, it was discussed to repurpose the focus group to present use cases for response. We will propose this to AHRQ & CDC later today on the call.
    • Possibly 2 focus groups would be conducted
      • Denver (Mile High APIC, a monthly meeting)
      • Salt Lake City
• Possible add-on/continuation of SSI detection work
  • Expanding the algorithm for other surgeries; e.g. colon, spine, breast
  • Implementation to other health systems (beta test), possible utilization of ACTION II network—Mayo, Providence, Baylor
• Task II
  • NSQIP data is not sufficient to train algorithm; small numbers.
    • The process to obtain national VA data is being expedited
      • It is not expected that a timeline extension is necessary; however, AHRQ and CDC will be updated if needs change.
• Task III
  • Dx vs. Hx
    • History data will be collected from diagnosis during the time period of the surgery.
  • Jef will finish writing the program and start running the Intermountain data-updating as necessary and following up with other organizations if inconsistencies are noted or changes are necessary.
    • Other organizations will pull the data and have it to Jef before Thanksgiving.
      • Vail will pull data after Denver Health has completed their pull
  • A table will be created to identify where each organization got the data for each risk factor
    • This information will be used to create a guide to aid the implementation at other organizations.
    • Scott will assess the marginal contribution of variables to assist other organizations in determining which/how many of the variables to include.
    • Possible publication (ICHE)-what are the risk factors for SSIs available from electronic data and the portability of our model across delivery systems.
  • Discussion regarding risk factor stratification
    • Some risk factors have a causal relationship with other factors
      • e.g., BMI will influence surgery duration
      • High risk factors may influence surgeon decision to operate
    • Patient modifiable vs. surgeon modifiable risk factors should also be considered.
• Draft report
  • The draft report is due February 15, 2011 (30 days prior to the final report being due)
    • Team members should keep the report in mind as they work on the project, building the report as you go by documenting all activities.
    • A team meeting is being planned for early 2011 to work on the final steps of the project and completing the draft report.
• Dissemination
  • Presenting at national meetings:
    • SHEA - April 2011 (abstract due Nov 7, 2010)
    • SIS - May 2011 (abstract due Dec 1, 2010)
    • Academy Health - June 2011 (abstract due Jan 13, 2011)
    • APIC - June 2011 (abstract due Jan 14, 2011)
  • A 4 month no-cost extension was discussed to fund travel/resources for meetings
    • Kendall will review extension request with AHRQ contracts.
  • Publications
    • Follow AHRQ guidelines
    • CDC guidelines are required only of CDC is an author
      • Teresa will verify CDC guideline rules
    • Draft titles for publication will be submitted to AHRQ and CDC for review
• Effective Use of Consultant Resources
  • Matt Samore will be most valuable in working on subtask 2.4 (Assess ability of electronic detection method to determine SSI rates with respect to estimates of annual national burden, and identifying SSI within healthcare facilities.
  • Amy Rosen-s time will be best used to review the draft report before submitting to AHRQ and CDC.

Noon - 1pm      Working Lunch & Discussion

1:00-2:00pm     Teleconference with AHRQ Project Officer and CDC Technical Advisors

• Provided task specific updates.
• Modification Request for Task 4: Given the delay of conducting the nursing focus group as originally planned in the proposal, it was discussed to repurpose the focus group to present use cases for response.
  • Based on OMB guidelines, 2 focus groups would be conducted
    • Denver
    • Salt Lake City
  • Proposals for the focus group will be submitted; we discussed the importance and need to comply with OMB guidelines. This will be submitted to Kendall at AHRQ for review.
• Possible add-on/continuation of SSI detection work discussed (see above). We will prepare a concept paper and submit to Kendall at AHRQ for review. Kendall noted that we did not need to use the ACTION II concept paper format.
• Dissemination—We inquired whether we had to comply with both AHRQ and CDC publication guidelines; Kendall will check and confirm, but believes AHRQ only unless a CDC staff person is a co-author; Teresa will verify CDC requirements. We will submit draft titles of papers. We also discussed possible presentation dissemination activities at national meetings and requested a 4-month, no-cost extension through July, 2011. We emphasized that all work will be completed on schedule, but additional time is requested to allow contract resources to be used to fund dissemination travel beyond the current end date—3/13/11. Discussed dissemination opportunities together with abstract due dates are:
  • APIC (abstracts due Jan 14; meeting in June)
  • SHEA (abstracts due Nov 7; meeting in April)
  • Surgical Infection Society (abstracts due Dec 1; meeting in May)
  • AcademyHealth (abstracts due Jan 13; meeting in June)

We discussed the value of reaching out to stakeholders at these various meetings. Kendall will check with AHRQ Contracts Office and let us know if we should proceed with a formal request for no-cost extension.

2:00-2:30pm Next Steps and Plans for Reporting & Dissemination

• Meeting wrap up, review of assignments and action plan.

2:30pm      Meeting Adjourned

Task Order No. 8
Improving the Measurement of Surgical Site Infection (SSI)
Risk Stratification and Outcome Detection
Team Meeting
Agenda

________________________________________________________________________________________________________________________________

27 January 2011
Vail Marriott
715 West Lionshead Circle
Vail, Colorado 81657

________________________________________________________________________________________________________________________________

28 January 2011
Vail Valley Medical Center
East Basement Room B
181 West Meadow Drive
Vail, Colorado 81657

Breakfast on your own—The First Chair Cafe in the Marriott opens at 6:30am

Task 1 —

SUSAN: Paragraph on modification to permit expenditures for dissemination.

ALL: Final invoices due to Denver Health within 15 days of contract end date?by March 28.

Task 2 —
The purpose of this task is to develop an SSI surveillance tool that enhances our ability to support current, manual detection process. We are targeting work savings, reduced burden. (Ref Scott's paper & VA example) Virginia requirement for statewide detection/reporting would require 160 IPs at a cost of $11.5 million. More than 50% of time is spent at the desk (Healther's article). The surveillance tool will enhance nurse work, moving them from infection counters to infection preventionists—frees nurses up to do more prevention. Cognitive surveillance support for human element (chart review, available electronic data, shoe leather).

• QA for current practice.
• Reduces burden of chart review (from 500:1 to 10:1); dramatic reduction in cost of chart review.
• Identify patterns of infection that might suggest opportunities for process improvement/re-engineering to enhance quality and safety.
• Change nature of the job.
• Meeting mandatory, hospital-wide reporting of SSI for value-based payments.
• Publicly available electronic surveillance tool vs. expensive, proprietary data mining surveillance tools like Theradoc, Medimine? That can cost up to $150,000, require a separate server, and have continuing maintenance/upgrade fees.

MAKOTO: adding text about algorithm, rationale for fitting on deep tissue/organ space infections, appendices (mapping ICD-CPT, calculation of variance, algorithm support)
MAKOTO/MIKE: logic of main 4 procedures selected
VA will send algorithm (4-5 variables) to systems to implement early next week.
ALL: Set up and run the algorithm.
ALL: Pull a random sample of 50 charts that are indicated + by algorithm to verify.
LUCY: will send cost capture template to document resource burden to set up & implement
ALL: complete the cost capture template as you set up and run the algorithm.
JEF: send SSI rate for Calendar Years 2007, 2008, 2009 to Makoto (# SSIs/# total surgeries)→Clarification, All Surgeries or Just Ones We Are Looking At? Will Rate Instability Be a Problem Where Small Numbers are Concerned, Thus Needing to Create 3-Year Rates?

JEF—complete frequency of surgeries table by system and year

JEF—complete frequency of SSIs table by system and year

LUCY: Get Juran quote—you can-t manage what you don-t measure

T2 Notes:

We err on the false positive side→suggested nursing focus group question on threshold of tolerance for misses (see Task 4).

To develop the surveillance algorithm, VA NSQIP data were used from 150 VAs across the U.S.; 70,000 cases. Randomly sampled 35,000 for training. Classification tree analysis using R. IP and OP data were used to develop our IP application. This may be an issue for fragmented systems of care (where the full continuum is not part of a single organization and/or data are not available on a single EMR platform); we expect some decrease in performance with fragments care where OP care occurs outside the system→follow on thought from Lucy-this may be remedied by meaningful use requirements and development of ACOs as called for in ACA legislation passed 3/23/10.
Post discharge surveillance is still a problem—this is KEY.

What is a clinically important infection? Superficial infections should not be included; it is difficult to define superficial infections and there is a lack of reliability in doing so. Most of the CABG infections are superficial. The CDC is moving towards a focus on deep wound and organ space infections.
DECISION: Tune for deep tissue/organ space infections; use 95% sensitivity.

NSQIP is a reference standard; we did discuss the heterogeneity in application of NSQIP. No method to measure SSI is perfect nor will there ever be a perfect measure. Disease is unknowable; the method is related to that unknowable target; the best we can get is consistent application of the method used. We are not suggesting validity of measure; we are comparing methods with unknown validity.

Document Resources needed:
EMR, electronic mechanism to gather data inputs at close to real time; dedicated data analyst (time?; approx. 2 weeks to set up with minimal implementation time); daily e-mails to IPs; SQL/SAS/R to run (NB: designed for portability and flexibility). Specify analyst skill requirements. Question on periodicity; Intermountain does daily; adopting facilities will need to determine what works best for them based on resources and needs.

We will estimate national SSI rate by applying the rate to total # of surgeries. Mike and Makoto will write up limitations and suggested approach for improved estimation.

Quality of HAI surveillance for required, publicly available data will likely suffer from the same problems Intermountain saw with their HAI and ADE surveillance systems—large increase in observations. Justifies a standardized electronic surveillance; a standard way of estimating rates on publicly available data. Problem with pure electronic surveillance (low specificity) overall higher rates.

Surveillance requirements are different than clinical action requirements.

Task 3 —

Clarified conditions and data availability for analyses

WALT: 3.1.1a expand

SCOTT: 3.1.1b add search terms and reference callouts with citations for key articles that led to additional risk factors

MIKE/MAKOTO: Description of VAMC data comparable to Denver Health on pg 11
JEF: Description of Intermountain comparable to Denver Health on page 11
HEATHER: Description of Vail data comparable to Denver Health on page 11

LIMITATION TO NOTE: Old NHSN used wound class, ASA score, duration of operation for all surgeries. We are now looking at surgery specific risk factor assessment. This is a limitation of our work. It is a watered down approach (e.g., differential impact of smoking on infection risk for hernia vs. CABG). We need to investigate risk factors as surgery specific. Now with more publicly reported data we can use that to get the needed large #s.

With a large dataset, you tend to keep a lot in a model; stepwise regression approach should lead to a more parsimonious model (5-10 outcomes/variables)

Task 4 —

LUCY to develop a fact sheet with example use case for nursing focus group

DECISION:
First focus group in Denver on February 3; schedule second to follow afterwards.
Lucy to listen in to Denver group (with participant permission).

Tool Sell-- The purpose of this task is to develop an SSI surveillance tool that enhances our ability to support current, manual detection process. We are targeting work savings, reduced burden. (Ref Scott's paper & VA example) Virginia requirement for statewide detection/reporting would require 160 IPs at a cost of $11.5 million. The surveillance tool will free nurses up to do more prevention. Change role from investigator to adjudicator; from infection control to infection preventionist. More time to prevent vs. counting beans; thus, using skills they were trained in to do best.

Use pilot analogy—Complementary decision support that does not replace the practice.

Acknowledgement of required human element: (1) expertise of IP, expert classification of presence of infection, professional value; (2) situation awareness to assess current state of the population (hospital, health care system); (3) investigational interactions.

T2 ? for nursing focus group: How many misses will IPs tolerate before they would not rely on such a tool? 10-1, 5-1?

How much would they value the data output (daily e-mail example from Scott's paper). Would you trust this information to avoid doing double work? Do responses vary by years in practice (we think so)?

Describe the burden of current human-long SSI surveillance practice.

Task 5 —

Reconciling relevant linkage across Task 2 and 3

Model for Continuous Quality Improvement in Reducing SSIs

*Patient, surgery, environmental

T2 measure surveillance allows you to determine if what you did as a result of T3 risk factor assessment is working or not.

Lessons Learned

Burden of unsupported practitioner SSI surveillance

Documented need for enhanced risk factor assessment (surgeon focus group) and receptivity of cognitive support from an electronic surveillance tool (nurse focus groups).

Explore natural language programming (NLP) and what could be added from text notes. New study shows that NLP will be more beneficial for the electronic identification of hospital-acquired wounds than bacteremias, UTIs, respiratory infections, etc. (Publication in progress).

Electronic algorithms don't perform the same.

Integrated medical records will have more utility. Post discharge surveillance is still a problem, requiring data from the full continuum of care (IP, OP).

The national SSI rate estimation is simplistic and limited. We will discuss how this could be optimally enhanced.

We are now looking at surgery specific risk factor assessment, accounting for differential impact of smoking on infection risk for hernia vs. CABG, for example. We need to investigate risk factors as surgery specific. Now with more publicly reported data we can use that to get the needed large #s.

Dissemination

ALL: Working paper titles and target journals sent to Jason.

MIKE: AMIA paper, 5-10 pages due March 17

3 conferences

AHRQ Annual Meeting