Background Report for the Request for Public Comment on Initial, Recommended Core Set of Children's Healthcare Quality Measures for Voluntary Use by Medicaid and CHIP Programs

Results

Overall Summary

In total, 24 measures are being recommended for the initial core set of health care quality measures for children. These include 13 measures of the quality of prevention and health promotion services, 5 measures of the quality of management of acute conditions; 4 measures of the quality of management of chronic conditions; 1 family experiences of care measure, and 1 availability (access) measure (Table 1). These represent a set of measures that address care across multiple settings (ambulatory [primary care, specialty care], emergency department (ED), and inpatient); multiple conditions (pregnancy, vaccine-preventable conditions, sexually transmitted infections, overweight, social and behavioral developmental delays, dental, appropriate antibiotic use for respiratory conditions, asthma, diabetes, and mental health); and multiple ages (in utero, neonatal, early childhood, school age, and adolescence). Taken together, the measure set is relevant to all 37.3 million children enrolled in Medicaid or CHIP who are potentially at risk for poor quality of care.¹¹ The lack of value represented by poor quality is a problem for the programs themselves and to society. While some measure topics address conditions with relatively low prevalence (e.g., children with diabetes, children with central lines^a hospitalized in intensive care units), the costs of inadequate or unsafe treatment are high. The well-child-care visit, general screening, and family experiences of care measures are relevant to all 37.3 million children.

Fourteen of the measures are currently NCQA HEDIS measures reported by Medicaid managed care plans, two are CMS-EPSDT program measures (both dental); three are measures submitted by individual State Medicaid or CHIP programs; one is a measure used by State Medicaid programs under a grant program; one is a measure derived from national vital statistics using State birth certificate data; one as nominated is stewarded by the California Maternal Quality Collaborative; and one is a measure used by the National Health Safety Network at the Centers for Disease Control and Prevention (CDC).

In terms of levels of evidence for underlying scientific soundness, two measures are grade A, 13 are grade B, 2 are both B and D; 1 is grade C, 4 are grade D alone, and 2 could not be graded. Given the well-known paucity of rigorous research on the effectiveness of services commonly provided to children¹² and to pregnant women, the number of measures with a Grade of B is impressive.

Eight of the measures as nominated require administrative data only, 10 currently use a hybrid of administrative and chart (medical records) data, one uses a combination of Medicaid enrollment data and State birth certificate data, one uses survey data, and others use other combinations. Information on the sources of data for some measures was not provided.

^a A central line allows concentrated solutions to be infused with less risk of complications. It permits monitoring of special blood pressures including the central venous pressure, the pulmonary artery pressure, and the pulmonary capillary wedge pressures. The central line can be used for the estimation of cardiac output and vascular resistance. The near end of the catheter may also be connected to a chamber for injections given over periods of months. A central line saves having to have frequent small injections or "drips" placed in the arms. A central line may also allow a patient to have medicine or fluids at home instead of in the hospital. 

General Themes

When considering the measures individually and as a group, it is critical to note a number of themes identified by the SNAC, Medicaid and CHIP officials in the listening session, and the public.

Importance of all measures considered

First, all measures considered, including the measures that were not included in the final SNAC and HHS recommendations, are important from some perspective. There were very few measures that were considered relatively unimportant by the SNAC on average (Appendix B). Second, decisions for inclusion were based primarily on considerations of validity and feasibility for use.

Size of the initial, recommended core set

In relation to the multiple health care needs of children and the CHIPRA legislation measurement domains, the initial core set of 24 measures seems relatively small. To many who are being asked to implement the measures, the number seems large. It is clear that implementation will not be automatic, and the number of measures in the recommended initial core set should be considered in the overall context of the CHIPRA legislation and other legislative and Federal Executive Branch initiatives, as well as in the context of the economic crisis faced by most States.¹³

Clear needs for technical assistance and time for implementation

By law, measures in the core set are intended for voluntary use; States, health plans, and providers are not required to use them. CMS will assist States by identifying standard measure specifications and providing focused, tailored technical assistance on information systems and measurement. States will receive a matching Federal Medical Assistance Percentage (FMAP) for the "effective collection and reporting of measures." With the implementation of the American Recovery and Reinvestment Act of 2009 (ARRA) for health IT adoption, State concerns about the burden of using medical records for quality measurement should lessen over time. Eligible providers will receive ARRA incentive payments for "meaningful use" of measures and information technology. Additional incentives for use of a core set with standardized specifications will include the ability of States to benchmark their own performance against aggregated data. Some States are already using many of the measures identified for the initial core set. Inevitably, implementation of the core measure set will take place over time, using a carefully staged process.

Improvements to the measure set are needed to meet legislative requirements

Implementation of the set "as is" will not be sufficient to achieve the vision of a comparable, evidence-based, understandable set of measures that can identify racial and ethnic, socioeconomic, and health condition disparities in health care for children. In making its recommendations for the initial core measure set, the SNAC emphasized that the measures would need to be reconfigured to be able to reflect children's health care quality across all Medicaid and CHIP programs, providers, consumers, and intermediaries (e.g., health plans contracting with State Medicaid programs). Modifying the measures for a more comprehensive set of programs and beneficiaries will take time and resources.

One example of the need for modification is the NCQA HEDIS measures, which comprise the bulk of the initial, recommended core measure set. NCQA HEDIS measures are currently used for reporting by Medicaid Managed Care plans, and may not be used for children enrolled in CHIP or children in States with limited or no Medicaid managed care. In 2008, 51% of Medicaid and CHIP child enrollees (17 million Medicaid enrollees and 5.3 million CHIP) were enrolled in managed care organizations.¹¹ Many States have no managed care presence.^b In some States, children with standalone CHIP coverage are not enrolled in managed care, but children in CHIP Medicaid expansion programs are. The highest number of plans and States reporting to NCQA is 163, across 32 States.¹⁴

Second, other measures—specifically those using data from State birth certificates reported to the National Vital Statistics System (NVSS), those used by non-government entities (e.g., California Maternal Quality Collaborative), and those currently in use by selected States—vary across States in measurement methodology or do not appear to be in use by States as currently specified.

Paucity of reporting to identify disparities

Few of the proposed measures are currently used to report data that distinguish care quality by race, ethnicity, tribe, socioeconomic status, or special health care need status among children, all of which are required by the CHIPRA legislation and are critical in light of the demographics of the U.S. child population.^15-16 Attention to improving the capacity of measures and datasets to assess disparities is needed.

Duration of enrollment considerations

For all measures, a common duration of enrollment calculation is essential to make valid and reliable assessments of health care quality in programs and populations and comparisons across and within institutions, programs, and States.

Need to specify accountable entities

Additionally, the SNAC agreed that it is critical to identify the appropriate entities that should be held accountable for health care quality in the multi-layered approach to care delivery (e.g., providing the service, facilitating care delivery).¹⁷

Need to develop and enhance children's health care quality measures

The initial core set and its voluntary implementation provide starting points for most intents and purposes. The recommended list (and the compilations of measures from which it was drawn) represents a significant achievement of a number of persistent and talented communities: those intensely engaged in measuring and improving health quality; child health researchers; child health advocates; Federal programs that supported and disseminated children's health care quality measures; and many others. The measures available today were developed and implemented without significant support. Medicaid and CHIP programs have traditionally not been able to acquire the significant resources needed to measure and improve health care for any of their populations. Thus, in addition to the work needed on the initial core set, there is a need to develop measures in several areas. The SNAC noted in particular the need for additional measures of specialty care, inpatient care, substance abuse care, and mental health treatment, as well as measures that link mainstream clinical care with other services that children receive (i.e., coordination of care), health outcome measures, and measures of the medical home.

Fortunately, CHIPRA provided support for advancing and improving pediatric quality measures and called for priorities to be set to guide a new pediatric quality measures program. States are also encouraged to experiment with and evaluate the core measure set using funds for the CMS demonstration projects.¹⁸

^b Arkansas, Idaho, Louisiana, Maine, Mississippi, Montana, North Carolina, North Dakota, South Dakota, Vermont, West Virginia.

Return to Contents

Recommended Measures

This section describes each of the recommended measures, organized by the measurement area specified by the CHIPRA legislation: prevention and health promotion, acute care, chronic care, family experiences of care, and availability of services. Although authors commissioned by AHRQ and CMS made progress in the areas of most integrated health care setting and duration of enrollment, it was not possible to identify any valid measures that were ready for immediate use for these topics. These are areas ripe for further development and testing.

Each description addresses the following, to the extent information is available: numerator and denominator of the measure (including requirements for continuous enrolment if applicable); levels at which the measure is specified and reported (e.g., health plan, provider, State); current reporting of the measure by number of entities (e.g., health plan, State); evidence level grade; SNAC voting results; information pertaining to the importance of including the measure in an initial core measure set (e.g., prevalence of the condition, size of the affected population, costs to Medicaid, current performance, and variations in performance), and challenges to implementation of the measure. In addition, NCQA has kindly provided information on specifications for HEDIS measures. 

Prevention and Health Promotion

Services to promote healthy birth, prevent prematurity, and prevention/health promotion for pregnant adolescents enrolled in Medicaid and CHIP

In requiring measures suitable for assessing the quality of preventive and health promotion services, CHIPRA noted a particular need for measures of the quality of services in the perinatal period that would promote healthy birth and prevent prematurity. Although more research on causal linkages is needed, prematurity is a growing problem in the United States. Maternal behaviors such as smoking affect gestational age and other aspects of infant development; smoking cessation counseling during prenatal care is recommended.¹⁹ In addition, perinatal services can affect the health of pregnant women; thus, some measures below should be reported separately for pregnant adolescents.

1. Timeliness of prenatal care. This measure assesses the number of pregnant women who had a prenatal visit in the first trimester or within 42 days of enrollment in the organization. Only women who delivered live babies and who were continuously enrolled between 43 days prior to delivery through 56 days after delivery are counted in the denominator. Current reporting levels are for health plans. Data can be stratified by women's age groups.

   The rationale behind the measure is that early prenatal care (in the first trimester of pregnancy) is that it helps prevent premature birth and other infant health problems. In addition, prenatal care is designed to improve the health of women during pregnancy.

   The measure has evidence grades of B and D. The SNAC voting process resulted in a rank of 9 for this measure. The data sources for this measure are administrative and medical records. The measure is currently in use by 158 Medicaid managed care (MMC) health plans across 32 States.

   The measure is potentially important. In 2007, Medicaid was the expected payer for 2,131,852 (or 42.14 percent of all) discharges related to the Major Diagnostic Category 14 (MDC 14), "pregnancy, childbirth, and the puerperium," with total charges of $24 billion, according to the Healthcare Cost and Utilization Project (HCUP).²⁰ About $1.3 billion of these Medicaid charges were accounted for by women < age 17 (123,000 discharges). Performance data reported by NCQA as valid indicate that an average 81.37 percent of adolescent women in reporting MMC health plans received timely prenatal care. Lower scoring plans reported an average receipt rate of 67 percent, and higher reporting plans reported an average rate of 92 percent, suggesting considerable variation across plans.

   To enable all State Medicaid and CHIP programs to report on this quality measure across all populations and programs, several challenges will need to be met: (1) specifying a reasonable enrollment and coverage duration for the denominator, and (2) improving the capacity of administrative or other electronic health record (EHR)-extractable data to make collection more feasible. Improving the inter-operability of data across the different settings where pregnant women may seek care will also improve the validity and reliability of performance reports.

   Other concerns include the underlying scientific soundness of the measure. An extremely well-cited and reputable report in the early 1980s estimated that $3 could be saved with every $1 spent on prenatal care.²¹ However, since that time there have been studies that question the use of prenatal care overall as an evidence-based quality measure. One reason is that factors other than health care services can affect pregnant women's and infants' health outcomes, and many believe that factors beyond the control of the health care delivery system are more important than the timing, amount, and quality of health care delivered during pregnancy. From some perspectives, starting to provide health care during pregnancy is too late, and attention should be paid to preconception care. Rigorous evidence reviews have found that some services provided during prenatal care are effective and should be recommended, and others are not recommended based on the available evidence (per the U.S. Preventive Services Task Force, USPSTF).²² Still other specific services routinely provided do not have a sufficient evidence base.²² The current consensus is that an optimal set of prenatal care measures would focus on the content of prenatal care (i.e., the extent to which evidence-based services are received). In addition, research is also needed on the effectiveness of other services for which evidence is currently insufficient and to determine the optimal timing and frequency of prenatal care for pregnant women.

2. Frequency of on-going prenatal care. This measure ascertains the number and percent of females in the denominator who had an unduplicated count of less than 21 percent, 21-40 percent, 41-60 percent, 61-80 percent, or more than 81 percent of expected visits, adjusted for the month of pregnancy at enrollment and gestational age. The denominator is the number of Medicaid-enrolled females who delivered a live birth on or between November 6th of the year prior to the measurement year and November 5th of the measurement year.

   This is a process measure with evidence grades of B and D. The measure was ranked number 2 in SNAC voting.

   The measure is currently well-specified for health plan and provider reporting and is currently validly reported to NCQA by 94 MMC health plans across 28 States, using the enrollment denominator of delivery of a live birth on/between November 6th of the year prior to the measurement year and November 5th of the measurement year. Current data sources for this measure are administrative and medical records.

   The measure can be considered important to stimulating future Medicaid and CHIP efforts to improve the health status of infants and adolescents, based on health care utilization and quality performance data. In 2007, Medicaid was the expected payer for 2,131,852 (or 42.14 percent of all) discharges related to the major diagnostic category of "pregnancy, childbirth, and the puerperium," with total charges of $24 billion.²⁰ About $1.3 billion of Medicaid charges were accounted for by women ages <17, who had 123,000 discharges in this category. NCQA reports that in 2007, 60 percent of pregnant women of all ages in the reporting health plans got 81 percent or more of the recommended number of prenatal visits. As with the measure of timeliness of prenatal care, in order to enable all State Medicaid and CHIP programs to report on this quality measure for all covered populations, several challenges will need to be met: (1) specifying a reasonable enrollment and coverage duration for the denominator, and (2) improving the capacity of administrative or other EHR-extractable data to make collection more feasible. Improving the inter-operability of data across the different settings where pregnant women may seek care will also improve the validity and reliability of performance reports.

   Other major concerns include the underlying scientific soundness of the measure. These concerns are similar to the issues discussed under Timeliness of Prenatal Care. The current consensus is that an optimal set of prenatal care measures would focus on the content of prenatal care (i.e., the extent to which evidence-based services are received). In addition, research is also needed on the effectiveness of other services for which evidence is currently insufficient and to determine the optimal timing and frequency of prenatal care for pregnant women.

3. Percent of live births weighing less than 2,500 grams. This is a measure derived from State-reported birth certificate data compiled in the National Vital Statistics System (NVSS) of the United States. The measure assesses the number of resident live births less than 2,500 grams as a percent of the number of resident live births in the State reporting period. Data are also available on very low birth weight (under 1,500 grams) babies.

   This is an outcome measure with an evidence level of grade B. SNAC voting resulted in a ranking of 10 for this measure.

   Low birth weight (often used as a proxy for premature birth) is an important condition that is highly prevalent, with disparities across States and across racial, ethnic, and income groups. Low birth weight is an important predictor of health outcomes for infants, and it is costly to the health care system and society. In 2006, 8 percent of babies weighed less than 2,500 grams at birth (345,369 babies). The rate was higher among babies born to non-Hispanic black women (14 percent). The 2010 Healthy People target is 5% or less.

   National data are not available on the number of these births to women enrolled in Medicaid or CHIP, although some States link NVSS data to Medicaid enrollment data. In 2007, Medicaid was the expected payer for 49 percent of hospital discharges (11,000 discharges) for Clinical Classification Software (CCS) principal diagnosis category "short gestation, low birth weight, and fetal growth retardation," at a total cost of $1.5 billion.

   An advantage of using State and national vital statistics data for this measure is that data on the race and ethnicity of the mother are available, which would help permit an assessment of disparities as directed by the CHIPRA legislation. Socioeconomic status could be derived from the ZIP code of the mother's residence, although this method has its limitations. However, State Medicaid and CHIP officials report that they will face challenges in reporting this measure because it is not readily obtainable from Medicaid and CHIP administrative data, and States currently linking NVSS to Medicaid data use different methods. A standardized approach would need to be agreed upon to ensure that data are comparable across State Medicaid and CHIP programs. A standardized approach would also be needed if a Federal agency were to be made responsible for reporting on the measure for Medicaid and CHIP enrollees.

   As discussed for the measures of prenatal care above, one of the challenges of this and other outcome measures is uncertainty about the extent to which the health care delivery system, and prenatal care services specifically, should be held accountable for poor results.

4. Cesarean rate for low-risk first birth women. This measure assesses the percent of women who had a cesarean section (C-section) among women with first live singleton births (also known as nulliparous term singleton vertex [NTSV] births) at 37 weeks of gestation or later.

   This is a process measure with grade B evidence. SNAC voting resulted in a ranking of 16 for this measure. The measure nominated is the one in use by the California Maternal Quality Collaborative (CMQC).

   Concern has been expressed recently about rising rates of cesarean deliveries with no indicated risk. C-sections have been associated with harm to the newborn as well as to the mother. The overall rate of C-sections rose steadily from 20 percent of all births in 1996 to 30.3 percent in 2005.²³ Those who have used the CMQC measure find high rates of NTSV C-sections (10-22 percent) and wide variation among hospitals (10.5 percent-30.2 percent in a large health care delivery network, and 10.3 percent -34.2 percent across 40 Arizona hospitals).^24-25 Approximately one-third of all hospital discharges for all C-sections have Medicaid as an expected payer;^24-25 the cost to Medicaid for all C-sections was $9.4 billion for 582,167 C-sections in 2007. Of these, 23,834 discharges were among women less than 17 years old, at an aggregate cost of $400 million.

   Not every State has a Maternal Quality Collaborative to collect the data. However, if every State collected the data as recommended on the U.S. Standard Certificate of Live Birth, and standards were established for linking to Medicaid and CHIP enrollment data, State programs could report the data at the State level. National Provider Information (NPI) on birth attendant and insurance data is also recommended to be collected on the standard birth certificate. Specifications are also available for using hospital discharge data.

Immunizations

Immunizations are a well-known approach to protecting children and the general population from infectious disease. The initial core set includes measures of immunization status for children at age 2 and adolescents at age 13.

5. Childhood Immunization Status. This NCQA HEDIS measure assesses the extent to which children who have reached their 2nd birthday (and have been continuously enrolled for 12 months prior to the birthday) received four diphtheria, tetanus, and acellular pertussis (DTaP) vaccinations; three polio (IPV); one measles, mumps and rubella (MMR); two H influenza type B (Hib); three Hepatitis B (HepB); one chicken pox (VZV); and four pneumococcal conjugate vaccines. (Variations by type of vaccine are also reported.)

   This is a process measure with an evidence grade of B. SNAC voting resulted in a ranking of 1 for this measure. Current data sources for reporting to NCQA are administrative and medical records. The measure is currently reported to NCQA by 163 MMC health plans in 32 States.

   Providing immunizations to children is an important public health strategy. All 2-year-olds enrolled in Medicaid and CHIP should have up-to-date immunizations. The national mean among health plans submitting valid reports to NCQA in 2007 was 65.4 percent (for Composite 3). Plans in the Northeast region of the United States have reported higher rates than plans in the South Central region.²⁶ Consistent with the NCQA 2007 data, a 2002 survey by the American Academy of Pediatrics (AAP) found that immunization coverage was 60 percent for preschool children in Medicaid/CHIP, 82 percent for those with other insurance, and 66 percent for uninsured children.²⁷

   Challenges to having valid immunization rates across Medicaid and CHIP programs and populations include the fact that immunizations are provided under several programs that provide services outside the primary health care setting, including the Vaccines for Children program.²⁸ Survey data collected by the National Immunization Program at the CDC, using a larger age range (19-36 months) finds higher overall rates of children up to date on their immunizations (80 percent overall), although this is a slightly different set of immunizations than used in the NCQA Composite 3 measure.²⁹ In the National Immunization Survey (NIS), lower rates occur among children with family incomes that are negative/poor, near poor/low, and middle compared to children with family incomes that are high, using the standard Federal Poverty Level rate, as well as between black and white children. Survey data linking insurance source to young children's up-to-date status on immunizations have not yet been reported publicly.

6. Immunizations for Adolescents. In 2010, this revised NCQA HEDIS measure will be used to assess the extent to which adolescents who are 13 years of age during the measurement year (and have been continuously enrolled for 12 months prior to the measurement year) had one dose of meningococcal vaccine (MCV4) and one tetanus, diphtheria, and acellular pertussis vaccine (Tdap) or one tetanus, diphtheria toxoids vaccine (Td).

   The evidence level for this measure is grade B. The SNAC voting process ranked the measure at 7. The current reporting specifications are for health plans and providers. In 2007, 22 of 36 State programs with managed care organizations/health insuring organizations (MCOs/HIOs) said they measured adolescent immunizations.^c For the NCQA measure, the data sources are administrative and medical records. The new specifications would permit access through electronic medical records.

   Because this is a revised measure for implementation in 2010, there are no performance data from NCQA. However, CDC's National Immunization Survey-TEEN is now collecting information on health insurance source and reports that 40.6 percent of publicly insured teens had both the appropriate doses of Tdap vaccine since 10 years of age and the appropriate dosage of meningococcal vaccine since 2 years of age.^30,d The rate for teens covered by private health insurance was approximately the same (41.0 percent). The most substantial difference was among teens not covered by health insurance; 23.1 percent of these teens were up to date on both vaccines.

   The Advisory Committee on Immunization Practices (ACIP) recommends one dose of Tdap (or Td) immunizations and one dose of MCV4 for 11-12 year-olds. Preventing pertussis in adolescents would reduce disease among that population and perhaps others by eliminating a reservoir of the disease. In 2007, 10,454 cases of pertussis were reported across all age groups, 2,650 of which were among 5-14 year-olds and 1,694 of which were among 15-24 year-olds.³¹ Pertussis symptoms can last for months and lead to school absences, but the condition is not life-threatening, and long-term effects are rare. Meningococcal disease, on the other hand, can be deadly or debilitating; 312 cases among 5-24 year-olds were reported in 2007.³¹ Current performance data for this measure are not available from NCQA. Jain and Hennessey reported a 27.2 percent Hib immunization rate for publicly covered 13-14 year-olds in 2006.³²

^c The data are not from NCQA, so it is unclear if the NCQA measure at the time was used by these States. The data are from a CMS survey.
^d The Tdap measure includes Tdap, Td, and Td/Tdap containing vaccine, unknown subtype. Meningococcal vaccine includes Menactra, Menomune, and meningococcal-unknown vaccine type.

Return to Contents
Proceed to Next Section