What We Learned
Learn about the CHIPRA quality demonstration States' strategies, lessons learned, and outcomes from these National Evaluation products:
- Final Summary
- Final Project Report
- Special Innovation Features
- State Spotlights
- Evaluation Highlights
- Implementation Guides
- Publications
- Presentations and Webinars
- Additional Resources
Final Summary
This summary, which draws from products produced throughout the evaluation, highlights program objectives, the strategies States used, and the lessons learned. To illustrate some of the lessons learned, this summary also includes short descriptions of selected activities implemented by the CHIPRA quality demonstration States. (PDF, 1.31 MB; HTML)
Final Project Report
The Final Project Report summarizes the 5-year CHIPRA Quality Demonstration Grant Program and national evaluation. (HTML)
Special Innovation Features
These two Special Innovation Features highlight two projects supported by the CHIPRA quality demonstration that developed and tested new electronic screening tools.
Electronic Student Health Questionnaire (eSHQ) enhances risk assessment for adolescents (PDF, 451 KB)
Developed under Colorado and New Mexico’s CHIPRA quality demonstration grant, the eSHQ is a computer tablet-based risk screening instrument. Providers who used the eSHQ reported it produced more complete risk data and facilitated referrals between primary care and behavioral health services. The eSHQ generated school-based health center (SBHC)-level and statewide reports for all students who used it. These reports allowed comparisons across SBHCs within a State and across the two States to help improve population health management and advocate for support of SBHCs.
Introducing Electronic Screening Tools for Developmental Delay and Autism into Pediatric Primary Care (PDF, 319 KB)
Screenings for developmental delays and autism in primary care pediatricians’ offices are an important tool for early identification, intervention, and referral to appropriate services. As part of Pennsylvania’s CHIPRA quality demonstration, The Children’s Hospital of Philadelphia (CHOP) implemented electronic screening in 12 of its pediatric primary care sites between 2011 and 2013. Adoption of electronic screening methods for developmental delay and autism helped improve efficient documentation and tracking of screening rates. The rapid availability of screening results, along with referral tools and other decision-support tools that are embedded in the Electronic Health Record (EHR), may also increase timely access to early intervention services.
State Spotlights
State Spotlights highlight the projects completed by the 18 States supported by the CHIPRA Quality Demonstration Grant Program. Each Spotlight describes the State’s strategies, lessons learned, and outcomes and can be helpful to other States interested in improving the quality of care for children.
Evaluation Highlights
The national evaluation team produced a series of issue briefs that frame the most pertinent evaluation findings for a variety of audiences concerned with children's health care quality. These 4-6 page briefs, entitled Evaluation Highlights, include both descriptive and analytic findings depending on the topic and time period of the demonstration.
Evaluation Highlight No. 13: How did CHIPRA quality demonstration States employ learning collaboratives to improve children’s health care quality?
This Evaluation Highlight is the 13th in a series that presents descriptive and analytic findings from the national evaluation of the Children's Health Insurance Program Reauthorization Act of 2009 (CHIPRA) Quality Demonstration Grant Program. This Highlight focuses on lessons learned from nine States—Alaska, Florida, Idaho, Maine, Massachusetts, North Carolina, Oregon, Utah, and West Virginia. These States implemented learning collaboratives and subsequently reported quantifiable improvements in medical home capacity and/or health care quality among the 137 child-serving primary care practices that participated in the CHIPRA quality demonstration. The analysis is based on work completed by States during the first 4.5 years of their 5-year demonstration projects.
This Evaluation Highlight includes a supplement that provides an overview of the learning collaboratives in each of the nine States, including their similarities and differences. [PDF (319 KB) or HTML]
Evaluation Highlight No. 12: How are CHIPRA quality demonstration States improving perinatal care?
Evaluation Highlight No. 11: How are CHIPRA quality demonstration States using quality reports to drive health care improvements for children?
Evaluation Highlight No. 10: How are CHIPRA quality demonstration States testing the Children's Electronic Health Record Format?
Evaluation Highlight No. 9: How are CHIPRA quality demonstration States supporting the use of care coordinators?
Evaluation Highlight No. 8: CHIPRA quality demonstration States help school-based health centers strengthen their medical home features
Evaluation Highlight No. 7: How are CHIPRA quality demonstration States designing and implementing caregiver peer support programs?
This Evaluation Highlight includes a supplement that describes the features of peer support programs across the States. PDF - 156.11 KB or HTML.
Evaluation Highlight No. 6: How are CHIPRA quality demonstration States working together to improve the quality of health care for children?
This Evaluation Highlight includes a supplement that describes the multi-State partnerships and the projects they are developing. PDF - 264.16 KB or HTML.
Evaluation Highlight No. 5: How are the CHIPRA Quality Demonstration States encouraging health care providers to put quality measures to work?
Evaluation Highlight No. 4: How the CHIPRA quality demonstration elevated children on State health policy agendas
Evaluation Highlight No. 3: How are CHIPRA Quality Demonstration States working to improve adolescent health care?
Evaluation Highlight No. 2: How are States and evaluators measuring medical homeness in the CHIPRA Quality Demonstration Grant Program?
This Evaluation Highlight includes a supplement that provides information on two medical home measurement tools, the creation of a cross-State database, the methods for collecting the data analyzed in this Highlight, and explores the MHI-RSF in six demonstration States PDF - 164.77 KB or HTML.
The Medical Home Index-Revised Short Form (MHI-RSF) is a medical home assessment tool being used in the CHIPRA quality demonstration. PDF - 563.9 KB ; HTML; The Medical Home Index: Revised Short Form: Pediatric [PDF - 133.79 KB]
Evaluation Highlight No. 1: How are CHIPRA demonstration States approaching practice-level quality measurement and what are they learning?
This Evaluation Highlight includes a supplement that describes the practice-level reporting efforts of the States featured in this Highlight. PDF [229.19 KB] or HTML.
Implementation Guides
Designing Care Management Entities for Youth with Complex Behavioral Health Needs
This Implementation Guide provides information about the process of designing care management entities (CMEs), which are designed to coordinate services provided by the many State agencies that serve youth with complex behavioral health needs. By ensuring services are comprehensive but not duplicative, CMEs can improve outcomes for these youth and their families and lower costs to States. This guide may be helpful to States interested in implementing or improving CMEs and for county agencies if they are responsible for financing behavioral health or social services in the State.
Engaging Stakeholders to Improve the Quality of Children's Health Care
This Implementation Guide includes suggested steps and tips for implementing initiatives for improving child health care quality from the CMS-funded national evaluation of the Children's Health Insurance Program Reauthorization Act of 2009 (CHIPRA) Quality Demonstration Grant Program. This guide focuses on the stakeholder engagement efforts in Georgia, Idaho, and Massachusetts, and is designed to help State officials and other program administrators engage and partner with stakeholders in their own child health care quality improvement initiatives. It provides a five-step approach to engaging stakeholders, breaking down each step into smaller tasks with key points and resources to consider throughout the process.
Publications
The national evaluation team is producing several journal articles that synthesize findings from several States and address topics of interest to the community of pediatric health services researchers.
Association Between Practice Participation in a Pediatric-Focused Medical Home Learning Collaborative and Reduction of Preventable Emergency Department Visits by Publicly-Insured Children in Massachusetts
This article published online in the September/October 2018 Pediatric Quality & Safety discusses the findings from a study that examined the impact of practice participation in a pediatric patient-centered medical home learning collaborative on preventable emergency department visits among children in MassHealth (the Massachusetts Medicaid/Children’s Health Insurance Program).
After The Demonstration: What States Sustained After the End of Federal Grants To Improve Children's Health Care Quality
This article published online November 8, 2017 in the Maternal and Child Health Journal (PDF, 573 KB) describes the 115 potentially sustainable elements within States' CHIPRA demonstrations. The National Evaluation Team analyzed data from grantee reports and key informant interviews to assess sustainment outcomes and key influential factors (PDF, 77 KB).
What Factors Influence States’ Capacity to Report Children’s Health Care Quality Measures? A Multiple-Case Study
This article in the January 2017 Maternal and Child Health Journal (PDF, 308 KB) presents findings from a case study of four high-performing States—Illinois, Maine, Pennsylvania, and Oregon—to identify factors that influence the ability of State Medicaid agencies to report CMS's core set of children’s health care quality measures.
Primary Care Physicians' Experiences With and Attitudes Toward Pediatric Quality Reporting
This article in the November-December 2016 issue of Academic Pediatrics (PDF, 170 KB) assesses primary care providers’ experiences with and attitudes toward pediatric-focused quality reports and identifies associated physician/practice characteristics.
Perspectives on Implementing Quality Improvement Collaboratives Effectively: Qualitative Findings from the CHIPRA Quality Demonstration Grant Program
This article in the January 2018 Joint Commission Journal on Quality and Patient Safety (PDF, 520 KB) presents the results of interviews with grantees from 12 demonstration States that offered quality improvement collaboratives to more than 300 primary care practices. The interviews identified which aspects of these collaboratives were viewed by organizers and participants as working well and which were not.
Associations Between Practice-Reported Medical Homeness and Health Care Utilization Among Publicly Insured Children
An article in the May-June 2015 issue of Academic Pediatrics explores the relationship between patient-centered medical homes and health care use in three CHIPRA Quality Demonstration Grant Program States. Researchers from the National Evaluation Team used practice-reported PCMH assessments and Medicaid claims from child-serving practices in Illinois, North Carolina and South Carolina to estimate the linkage between medical homeness and well-child care and nonurgent, preventable, or avoidable emergency department (ED) use.
Nine States' Use of Learning Collaborative to Improve Children's Health Care Quality in Medicaid and CHIP
This article in the November 2013 Academic Pediatrics supplement on quality improvement examines how nine CHIPRA quality demonstration States are using learning collaboratives to improve health care quality for children. The various approaches, goals, and challenges described in the article offer lessons that other States may find useful in creating their own learning collaboratives.
Presentations and Webinars
June 2017 Presentation
This presentation by Paul Kirby of UMass Medical School was given at the June 2017 AcademyHealth Research Meeting. He presented findings from a study of the association between pediatric primary care practice participation in a learning collaborative designed to develop pediatric patient-centered medical home capacities and a reduction in preventable use of emergency department services by children enrolled in those practices, particularly children with chronic health conditions.
Winter 2016 Webinar Slides
On February 11, 2016, AHRQ and CMS presented a free Webinar, “Amplify Your Impact on Child Health Quality: Learning from the CHIPRA Quality Demonstration Grant Program." The presentations featured highlights and a discussion of promising practices and key lessons learned from the 5-year demonstration grant program to improve health care quality for children in Medicaid and CHIP.
The slide deck from that Webinar is available as an HTML and as a PDF (1.1 MB).
Fall 2015 Webinar Slides
In Fall 2015, staff from AHRQ and the National Evaluation Team presented a series of webinars on lessons learned from the CHIPRA Quality Demonstration Grant Program. These webinars focused on quality measurement and quality measure reporting and included discussants from CMS and the Demonstration States. The slide decks from those presentations are available as separate PDFs.
AcademyHealth State-University Partnership Learning Network (SUPLN) Web Conference: Findings from the CHIPRA Quality Demonstration Grant Program
The slide deck is available as an HTML and as a PDF (2.6 MB).
Putting Quality Measures to Work: Presentation for the Association of Medicaid Medical Directors - Lessons from the CHIPRA Quality Demonstration Grant Program
The slide deck is available as an HTML and as a PDF (838 KB).
Using Quality Measures to Drive Improvement: Presentation for the National Academy of State Health Policy’s Vanguard Network and CHIP Directors - Lessons from the CHIPRA Quality Demonstration
The slide deck is available as an HTML and as a PDF (246 KB).
Evaluation of the CHIPRA Quality Demonstration Grant Program: An Overview
On July 22, 2015, Henry Ireys presented this overview presentation via a Web conference to CMS’ Quality Technical Advisory Group (QTAG). The slide deck is available as a PDF (310 KB).
Associations Between Practice-Reported Medical Homeness and Health Care Utilization Among Publicly Insured Children
This presentation, given by Anna Christensen at AcademyHealth’s 2015 Annual Research Meeting, discusses the relationship between patient-centered medical homes and health care use in three CHIPRA Quality Demonstration Grant Program States. Researchers from the National Evaluation Team used practice-reported PCMH assessments and Medicaid claims from child-serving practices in Illinois, North Carolina and South Carolina to estimate the linkage between medical homeness and well-child care and nonurgent, preventable, or avoidable emergency department (ED) use. These findings were also published in the May-June 2015 issue of Academic Pediatrics. The slide deck is available as a PDF (126 KB).
Introducing Electronic Screening Tools for Developmental Delay and Autism into Pediatric Primary Care: Effects on Screening Documentation
This presentation, given by Brenda Natzke at AcademyHealth’s 2015 Annual Research Meeting, discusses how, as part of Pennsylvania’s CHIPRA quality demonstration, The Children’s Hospital of Philadelphia (CHOP) implemented electronic screening in 12 of its pediatric primary care sites between 2011 and 2013. Adoption of electronic screening methods for developmental delay and autism helped improve efficient documentation and tracking of screening rates. This screening intervention was also profiled in a Special Innovation Feature produced by the national evaluation team. The slide deck is available as a PDF (196 KB).
Improving Quality of Care for Children: Early Lessons from the CHIPRA Quality Demonstrations
Members of the national evaluation team presented at AcademyHealth’s 2014 National Child Health Policy Conference. Slides from each of the presenters from the national evaluation team are posted.
CMS CHIPRA Quality Demonstration Grant Program Webinar Series
In the summer and fall of 2013 CMS conducted a five-part webinar series to share early findings from the CHIPRA Quality Demonstration Grants. The first webinar provides an overview of the CHIPRA quality demonstration, while the other four describe the experiences of the grantees.
Presentation from the CMS Medicaid and CHIP Quality Conference, June 2012
On June 15, 2012, Henry Ireys presented an update of the national evaluation activities to the CHIPRA quality demonstration States at the second annual CMS Medicaid and CHIP Quality Conference in Baltimore, MD. The presentation provided updates on the national evaluation team's data collection activities, early observations from site visits, Web page updates, and next steps. The slide deck is available in HTML and as a PDF (229 KB).
Additional Resources
CMS Web site about the CHIPRA Quality Demonstration Grant Program
In February 2010, the Centers for Medicare & Medicaid Services (CMS) awarded 10 grants, funding 18 States, to improve health care quality and delivery systems for children enrolled in Medicaid and the Children's Health Insurance Program (CHIP). CMS provides overviews of each demonstration State's funding and projects on InsureKidsNow.gov.
CHIPRA Children's Health Care Quality Measurement and Improvement Activities
This Web page, hosted on AHRQ's Web site, provides information on the following activities: (1) identifying an initial core set of children's health care quality measures for voluntary use by Medicaid and CHIP programs, (2) implementing the CHIPRA Pediatric Quality Measures Program using grants and contracts, and (3) evaluating CMS's Quality Demonstration project awards and creating a Model Children's Electronic Health Record Format.
Please note: This Web site uses the term "national evaluation" to distinguish this evaluation of the entire demonstration program from evaluations commissioned or undertaken by grantees. The word "national" should not be interpreted to mean that findings are representative of the Nation as a whole.
