Assessing the Impact of Value-Based Purchasing Programs

“AHRQ’s value cannot be overstated. We know a lot of what works in basic and clinical sciences, but we don’t focus on what actually happens when we translate that knowledge into the bigger world. AHRQ is absolutely focused on what happens in the bigger world.”

The concept behind most value-based purchasing programs in healthcare seems simple: Reward the hospitals that meet certain quality-related performance goals and penalize those that fail to reach those targets. While linking payment to performance may be straightforward in theory, in practice, it’s anything but.

That’s why the work of AHRQ-funded researcher and health economist Teresa M. Waters, Ph.D., has been instrumental in building the evidence about whether these programs—and their multi-millions of dollars in  financial rewards and penalties—are based on sound, meaningful measures. Dr. Waters is professor and chair of the Department of Health Management and Policy at the University of Kentucky School of Public Health, Lexington.

When the Centers for Medicare & Medicaid Services (CMS) launched its initial value-based purchasing program (also known as pay-for-performance) in 2008, it was clear that a new era had begun. “It was really a sea change in the way we paid for care,” Dr. Waters recalled. “It had a lot of logic to it, but not a lot of strong evidence that it was going to do what we wanted it to do.”

Under the rule, known as the Hospital-Acquired Conditions Initiative, CMS denied additional Medicare payments to hospitals when patients developed a preventable in-hospital complication of their care. In 2011, Dr. Waters undertook an AHRQ-funded investigation that examined four of the eight conditions deemed as preventable.

Her study looked at rates of change for catheter-associated urinary tract infections (CAUTI), central-line bloodstream infections, (CLABSI) falls, and hospital-acquired pressure ulcers. Incident rates of change dropped by 11 percent for CAUTI and 10 percent for CLABSI but remained essentially the same for falls and pressure ulcers. The results, Dr. Waters said, likely reflected the early impact of hospitals’ targeted improvement efforts.

As more financially far-reaching initiatives from CMS followed, Dr. Waters used AHRQ funding to examine their impact on hospitals’ financial viability, including safety net hospitals that treat greater numbers of poor, high-need patients. CMS’ Hospital Readmissions Reduction Program (HRRP), launched in 2013, levied financial penalties on hospitals with unplanned readmissions for congestive heart failure, acute myocardial infarction (heart attack), and pneumonia. At the program’s outset, hospitals that exceeded their expected readmission rates were penalized up to 1 percent of their Medicare revenue. The HRRP has since expanded to include additional medical conditions and steeper financial penalties.

Safety net hospitals did, in fact, suffer higher penalties under the HRRP than non-safety net hospitals, Dr. Waters’ research found. However, their financial viability was not as threatened as some feared due to nonpatient care revenue.  Over time, though, this pattern could become a vicious cycle, she  said. “Safety net hospitals are improving, but they can’t improve enough to significantly reduce their penalty status, because everyone else is [also] improving, and they keep getting penalties.”

A related study from this AHRQ grant shed light on the role of unreliable readmission measures used to calculate hospitals’ payment penalties. “Getting penalties doesn’t necessarily give you information on how to improve care,” Dr. Waters noted. In particular, measures for heart attacks were found to be problematic and didn’t factor in the wide and unmeasured variation in patients’ underlying conditions.

“Many hospitals have developed continuity of care programs to reduce their readmissions,” Dr. Waters said. “Patients have high readmissions for a lot of reasons that even hospital systems with many community providers struggle to address.” Until payments can reflect differences in social determinants of health, “we may reach a ceiling on how much we can improve some of these outcomes measures.”

Dr. Waters’ current grant, expected to be completed in 2022, is assessing the joint impact of the HRRP and Hospital Acquired Conditions Reduction Program on quality and patient safety, using AHRQ’s inpatient quality indicators and patient safety indicators.

Research findings to date underscore her ongoing caution about how measures are calculated that have enormous financial impact. “The [CMS] program has modified the metrics used to assess penalties so many times that, in essence, hospitals have great difficulty responding.” CMS should not pay for poor quality of care, she emphasized, but should focus on how measures are developed and how often they are changed.

In addition to her academic appointment at the University of Kentucky, Dr. Waters is a member of the American Society of Health Economists, AcademyHealth, and the International Health Economists Association. Together with her collaborator, Dr. Michael Thompson, now at the University of Michigan, she received the HCUP Outstanding Article of the Year Award, awarded by AHRQ and AcademyHealth, for her 2016 article on readmission measures used by the HRRP program. Dr. Waters is a member of the editorial board for Medical Care Research and Review.

Principal Investigator: Teresa M. Waters, Ph.D.
Institution: University of Kentucky School of Public Health
Grantee Since: 2011
Type of Grant: Various

Return to AHRQ Grantee Profiles