Roadmap for Implementing Pediatric Quality Measures for Performance Improvement
This Roadmap serves as a guide for stakeholders planning to implement pediatric quality measures for performance improvement. It is intended for use by states, health plans, hospitals, and other providers embarking on a quality improvement project. The Roadmap provides overall guidance on key steps to consider when launching an improvement effort—from defining the project’s goal(s), to forming a team, to understanding context for quality improvement, through implementation and evaluation.
The Roadmap was developed as part of the Pediatric Quality Measures Program (PQMP), sponsored by the Agency for Healthcare Research and Quality (AHRQ) and the Centers for Medicare & Medicaid Services (CMS). The PQMP includes two phases: (1) development of new evidence-based pediatric quality measures, and (2) dissemination and implementation of the measures, with a focus on the feasibility and usability of the measures in real world settings. Using an implementation science approach, the second phase of the PQMP project seeks to understand the contextual factors—barriers or facilitators—that may impact measure uptake and quality improvement. Understanding how context plays a role in implementation can better inform the adoption of pediatric quality measures, with the ultimate goal of driving improvements in care for children.
Using the Roadmap below, follow the steps to explore the measure implementation process, including understanding contextual factors that may help or impede the project. Many of the steps include links to existing external resources. For specific information on implementing selected measures from the PQMP, refer to the PQMP Implementation Toolkits (Step 4).
Implementation Roadmap
Step 1: Defining Your Improvement Goal
Step 2: Forming an Implementation Team
Step 3: Understanding and Optimizing Your Context
Step 4: Implementing an Improvement Project
Step 5: Evaluating Implementation
Step 1: Defining Your Improvement Goal—Setting an Improvement Aim and Establishing Measures
A first step in any quality improvement effort is identifying areas for improvement based on an initial assessment and available data. Then, you must set a clear improvement goal that is specific and measurable, and based on an applicable and appropriate quality measure. Measures enable you to assess whether the changes being made are leading to improvement. Although the broader evidence base for pediatric care has historically been more limited than for adults, the PQMP has set out to increase the available measures for assessing pediatric care. For information on setting an aim and selecting pediatric measures for quality improvement, see the below links to relevant resources. An illustrative example of a PQMP measure and associated quality improvement goal is can be found below.
Example: PQMP NCINQ II Team
Resources
Setting an Improvement Aim
Establishing Measures
PQMP Resources
Step 2: Forming an Implementation Team
Based on your improvement goal, the next step is building a team that will carry out the project. An effective team will include a range of staff with expertise related to the area identified for pediatric quality improvement, as well as relevant stakeholders including those who would be using the given measure(s). The team may include operational leaders, clinicians, quality measurement and data experts, informatics and IT experts, administrative staff, case managers, health educators, payers, patients, children, youth, parents, families and other caregivers, and community representatives. For information on forming your team, see the below links to external resources. An illustrative example of a team from one of the PQMP projects is below.
Example: PQMP CEPQM Team
Resources
- Institute for Healthcare Improvement Resource: Forming an Implementation Team.
- AHRQ Practice Facilitation Handbook: Creating Quality Improvement Teams and QI Plans.
- Engaging Stakeholders to Improve the Quality of Children’s Health Care (PDF).
- Guide to Patient and Family Engagement in Hospital Quality and Safety.
References
Step 3: Understanding and Optimizing Your Context
Context plays an important role in the implementation process and can influence the degree of success achieved in a quality improvement project. Having an awareness and understanding of the contextual factors that may be at play, before initiating a project, allows you to plan for them by addressing any barriers or leveraging any facilitators. In order to better understand your context, see the link below to a self-study exercise on contextual factors. This exercise is based on the Model for Understanding Success in Quality (MUSIQ), which is a conceptual model for understanding and optimizing contextual factors affecting quality improvement efforts. Additional links to other relevant external resources are also included below, including other frameworks that can be used to assess contextual factors. An illustrative example of a contextual factor from one of the PQMP projects is also provided below.
Example: PQMP IMPLEMENT Team
Self-study Exercise on Contextual Factors Using the MUSIQ Framework
- This exercise, developed by Cincinnati Children's Hospital Medical Center, allows you to score a variety of factors related to your QI team, organizational support, and the external environment to identify areas in need of attention before QI efforts are started: Excel calculator for MUSIQ1.
Additional Resources
- Tailored Implementation for Chronic Diseases (TICD) Worksheet: Initial Assessment of Determinants (PDF).
- Consolidated Framework for Implementation Research (CFIR) Guide, including an Implementation Strategy Matching Tool to help guide choice of implementation strategies for mitigating identified barriers.
References
- Kaplan HC, Provost LP, Froehle CM, Margolis PA. The model for understanding success in quality (MUSIQ): building a theory of context in healthcare quality improvement. BMJ Qual Saf 2012;21(1):13-20.
- Reed JE, Kaplan HC, Ismail SA. A new typology for understanding context: qualitative exploration of the model for understanding success in quality (MUSIQ). BMC Health Serv Res 2018 Jul 25;18(1):584.
- Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009;4:50.
- Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, Baker R, Eccles MP. A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci 2013;8:35.
1. MUSIQ Excel Calculator, from Innovation Ventures at Cincinnati Children’s Hospital Medical Center, is licensed under CC BY-NC-SA 4.0 International.
Step 4: Implementing an Improvement Project
Once you are ready for implementation, there are a number of resources that can help with the process of deciding what changes you will make and how you will integrate those changes into your organization’s processes. This includes selecting changes; testing them, for example, through Plan-Do-Study-Act (PDSA) cycles; and then implementing them. For information on implementing specific pediatric quality measures that were created and tested as a part of the PQMP, see the link below to a set of PQMP implementation toolkits. Additional links to other more general resources on measure implementation are also included below. An illustrative example of the core toolkit elements from one of the PQMP projects is also provided below.
PQMP Implementation Toolkits
- Safe and Judicious Use of Antipsychotic Medications in Children and Adolescents.
- Pediatric Asthma Emergency Department Use.
- CAHPS Child Hospital Survey (Child HCAHPS).
- Quality of Pediatric Hospital-to-Home Transitions.
- Transcranial Doppler Ultrasonography (TCD) Screening Among Children with Sickle Cell Anemia.
Example: PQMP IMPLEMENT Team
Resources
- Institute for Healthcare Improvement Resource: Improvement Project Roadmap.
- Institute for Healthcare Improvement Resources: Selecting Changes; Testing Changes; and Implementing Changes.
References
- Glisson C, Schoenwald SK. The ARC organizational and community intervention strategy for implementing evidence-based children's mental health treatments. Ment Health Serv Res 2005 Dec;7(4):243-259.
- Hurlburt M, Aarons GA, Fettes D, Willging C, Gunderson L, Chaffin MJ. Interagency Collaborative Team Model for Capacity Building to Scale-Up Evidence-Based Practice. Child Youth Serv Rev 2014 Apr;39:160-168.
Step 5: Evaluating Implementation
A key tool for assessing performance in quality improvement is a statistical process control chart, which allows you to visually plot your performance data over time in a graph or chart to see and better understand any variation that may be occurring. For information on how to assess your quality improvement efforts using statistical process control tools, see the links below to external resources. An illustrative example of a statistical process control chart used in the PQMP is also described and provided below.
Additionally, to learn more about how or why an intervention did or did not result in improvements, a process or summative evaluation may be conducted. A process evaluation allows you to monitor the implementation process throughout the course of a QI initiative to assess how implementation is proceeding and if changes are being implemented as intended. A summative evaluation is planned throughout the project and focused on assessing if the planned changes were successful in changing outcomes, based on data collected during the program. Links to general external references on evaluation are included below.
Example: PQMP P-HIP Team
Resources
- Institute for Healthcare Improvement Resource: Videos on Statistical Process Control Charts (Part I and Part II).
- Overview of Types of Evaluation (CDC) (PDF).
References
- Grant A, Dreischulte T, Guthrie B. Process evaluation of the data-driven quality improvement in primary care (DQIP) trial: active and less active ingredients of a multi-component complex intervention to reduce high-risk primary care prescribing. Implement Sci 2017;12(1):4.
- Gupta M, Kaplan HC. Measurement for quality improvement: using data to drive change. J Perinatol 2020;40:962–971.
- Hulscher ME, Laurant MG, Grol RP. Process evaluation on quality improvement interventions. Qual Saf Health Care 2003;12(1):40-46 (PDF).
- Owen RR, Drummond KL, Viverito KM, et al. Monitoring and managing metabolic effects of antipsychotics: a cluster randomized trial of an intervention combining evidence-based quality improvement and external facilitation. Implementation Sci 2013;8:120.
- Scanlon DP, Alexander JA, McHugh M, et al. Summative evaluation results and lessons learned from the Aligning Forces for Quality program. Am J Manag Care 2016 Aug;22(12 Suppl):s360-372.
- Stephens, T.J., Peden, C.J., Pearse, R.M. et al. Improving care at scale: process evaluation of a multi-component quality improvement intervention to reduce mortality after emergency abdominal surgery (EPOCH trial). Implement Sci 2018; 13:142.
P-HIP (the Pediatric Hospital Care Improvement Project) sought to improve documentation and reporting on hospital-to-home transitions through timely and feasible data collection, uploading, and measure scoring.
Statistical Process Control Chart
The Pediatric Hospital Care Improvement Project (P-HIP) Collaborative collected Hospital-to-Home Transitions data from eight participating hospitals for nearly 4.5 years in order to analyze results over time. The analysis includes 24 months of data before the interventions were implemented to allow researchers to establish a baseline for comparison purposes.
The statistical process control charts that were used are available for site-specific results for each of the eight participating hospitals at this link and combined results for the overall collaborative are shown below and at this link.
