Meeting Minutes, November 2016
National Advisory Council
Contents
Summary
Call to Order and Approval of July 22, 2016, Summary Report
Director's Update
The Learning Health Care System
Further Discussion with the Panel of Speakers
Public Comment
Adjournment
Summary
NAC Members Present
Elizabeth A. McGlynn, Ph.D., Kaiser Permanente (Chair)
David J. Ballard, M.D., Ph.D., M.S.P.H., FACP, STEEEP Global Institute, Baylor Scott & White Health (via telephone)
Alice S. Bast, Beyond Celiac
Shari Davidson, National Business Group on Health
Jennifer E. DeVoe, M.D., D.Phil., M.Phil., M.C.R., Oregon Health & Science University
Robert S. Dittus, M.D., M.P.H., Vanderbilt University Medical Center (via telephone)
José Julio Escarce, M.D., Ph.D., University of California, Los Angeles
Mary Fermazin, M.D., M.P.A., Health Services Advisory Group, Inc.
Donald A. Goldmann, M.D., Institute for Healthcare Improvement, Harvard Medical School (via telephone)
Kevin L. Grumbach, M.D., School of Medicine, University of California, San Francisco (via telephone)
Mary D. Naylor, Ph.D., R.N., FAAN, University of Pennsylvania School of Nursing (via telephone)
Lucy A. Savitz, Ph.D., M.B.A., Intermountain Healthcare (via telephone)
J. Sanford Schwartz, M.D., University of Pennsylvania
Paul E. Sherman, M.D., M.H.A., CPE, FAAP, Group Health Physicians
Jed Weissberg, M.D., FACP, Institute for Clinical and Economic Review
Alternates Present
Naomi Tomoyasu, Ph.D., Department of Veterans Affairs (for David Atkins, M.D., M.P.H.)
Shari M. Ling, M.D., Centers for Medicare & Medicaid Services (for Patrick Conway, M.D.)
Charles J. Rothwell, M.B.A., M.S., National Center for Health Statistics, Centers for Disease Control and Prevention
AHRQ Staff Members Present
Andrew B. Bindman, M.D., Director
Sharon B. Arnold, Ph.D., Deupty Director
Jaime Zimmerman, M.P.H., PMP, Designated Management Official
Karen Brooks, CMP, NAC Coordinator
Call to Order and Approval of July 22, 2016, Summary Report
Elizabeth A. McGlynn, Ph.D., Chair of the National Advisory Council (NAC), Agency for Healthcare Research and Quality (AHRQ), called the group to order at 8:30 a.m. and welcomed the NAC members, other participants, and visitors, including viewers of the meeting’s webcast. She referred to the draft minutes of the previous NAC meeting (July 22, 2016) and asked for changes and approval. The NAC members voted unanimously to approve the July meeting minutes with no changes.
Dr. McGlynn announced that the following NAC members would be rotating off the Council following this meeting: David J. Ballard, M.D., Ph.D., M.S.P.H., FACP, Shari Davidson, Mary Fermazin, M.D., M.P.A., Ann L. Hendrich, Ph.D., R.N., FAAN, Paul E. Sherman, M.D., M.H.A.,CPE, FAAP, Patricia J. Skolnik, and Jed Weissberg, M.D., FACP. She noted that the meeting’s agenda would include two periods for public comment, and she asked the NAC members to introduce themselves.
Director's Update
Andrew B. Bindman, M.D., Director, AHRQ
Director Andy Bindman welcomed the NAC members, speakers, and other guests. He reviewed the day’s agenda, noting that the meeting’s presentations and discussions would focus on the theme of a Learning Health Care System.
Update and Agency Budget
Dr. Bindman reported that Dr. McGlynn recently became Vice President, Kaiser Permanente Research, and Executive Director, Center for Effectiveness and Safety Research.
In response to a discussion from July’s NAC meeting, AHRQ conducted a series of actions targeting the potential impact of Evidence-based Practice Center (EPC) reports. The actions included identifying key influencers within stakeholder groups who can amplify the reach of EPC reports, the beginning of a process to obtain feedback on the usefulness of reports, and the start of an effort to encourage health systems to identify opportunities for the development and use of EPC reports. The agency also has established a quality improvement team to address efficiency of the EPC review process and selection process.
In response to another discussion from July’s NAC meeting, AHRQ began to assess existing quality measurement activities and to identify potential partners who can perform maintenance activities. The agency also is being mindful of new sources of data, especially related to electronic health records (EHRs).
In response to another discussion from July’s NAC meeting, AHRQ developed a charter for a new subcommittee that will provide advice to the NAC chair on issues relating to the coordination of AHRQ’s work on patient-centered outcomes research (PCOR) and that of other Federal agencies, including the PCOR Institute (PCORI). Mary D. Naylor, Ph.D., R.N., FAAN, will chair this new subcommittee of the NAC (SNAC). Invitations have been forwarded to potential members.
In September, AHRQ presented a Diagnostic Errors Research Summit, with 200 attendees in-house and 400 watching on the Web. The summit brought together an international group of expert members of the diagnostic community to discuss efforts to reduce diagnostic error. It featured presentations on the work of AHRQ, on the history of the Institute of Medicine (now titled the National Academy of Medicine), and on diagnostic tools. Next steps include the development of research (funding mechanisms) to understand and improve diagnostic safety and to create new tools to improve it.
Dr. Bindman reported that AHRQ’s support for investigator-initiated and targeted research spans the agency’s program areas. Total grant funding in these areas of research for FY 2017 is estimated to be $192.3 million. Total grant funding for FY 2016 was $189.6 million.
The FY 2017 discretionary budget request for AHRQ is $363.7 million. That is an increase of $29.7 million over the FY 2016 discretionary budget. The total FY 2017 program-level budget request is $469.7 million. The total program-level budget includes $106 million in mandatory funds from the PCOR Trust Fund. On June 7, the Senate Subcommittee on Labor, Health, and Human Services recommended $324 million for FY 2017, and on July 14, the House Appropriations Committee recommended $280.2 million for FY 2017. AHRQ currently is providing only necessary and limited funding in order to provide for the continuation of projects and activities.
As AHRQ Director, Dr. Bindman has been informing Congress on the agency’s activities and has been working to align the agency’s work with the work of the National Institutes of Health (NIH), Centers for Medicare & Medicaid Services (CMS), and PCORI. He is overseeing the agency’s internal review of activities and their impact. That serves a goal of continual improvement within AHRQ, realizing program effectiveness, program spread, and AHRQ’s contribution to improved health care.
How AHRQ Makes a Difference
Dr. Bindman stated that AHRQ invests in research and evidence to understand how to make health care safer and improve quality. It creates materials to teach and train health care systems and professionals to catalyze improvements in care. It generates measures and data used to track and improve performance and to evaluate progress. Dr. Bindman presented the following agency updates:
Research and Evidence
- The current overall rate of successful applications for AHRQ research grants is 28 percent. That includes a 42 percent success rate for F32 training grants, a 33 percent success rate for R01 investigator-initiated research grants, and a 15 percent success rate for R18 dissemination grants.
- Recent funding announcements include an R18 grant for Large Health Services Research Demonstration and Dissemination Projects for Prevention of Healthcare-Associated Infections and an R01 grant for Large Research Projects for Prevention of Healthcare-Associated Infections.
- AHRQ, collaborating with CMS, awarded $13 million in grants for the testing of pediatric quality measures. Six grantees will further develop the measures, testing them in real-world settings and assessing the feasibility of use for Medicaid and the Children’s Health Insurance Program (CHIP).
Tools and Training
- AHRQ produced a Healthcare Simulation Dictionary, featuring more than 100 terms and definitions to enhance communications and clarity for users of health care simulation in education, assessment, and research. The document was a collaboration with the Society for Simulation in Healthcare.
- The U.S. Preventive Services Task Force (USPSTF) produced final new recommendations for screening for skin cancer, screening for lipid disorders in children and adolescents, screening for latent tuberculosis infection, and primary care interventions for breastfeeding. Topics addressed in draft recommendations were screening for genital herpes infection, screening for preeclampsia, and screening for obesity in children and adolescents.
- The EPCs produced systematic reviews for topics including strategies to de-escalate aggressive behavior in psychiatric patients, patient safety in ambulatory settings, and omega-3 fatty acids and cardiovascular disease.
- AHRQ and PCORI have begun a process to create a memorandum of understanding concerning the creation and updating of EPC reviews.
Data and Methods
- The State Snapshots from AHRQ’s 2015 National Quality and Disparities Report indicated that Maine, Massachusetts, Wisconsin, Minnesota, and New Hampshire were the Nation’s best performing States for health care quality.
- AHRQ’s Chartbook on Care Affordability indicated that the percentage of people who reported difficulty in paying their medical bills decreased from 21 percent in 2011 to 17 percent through the first half of 2015.
- The Medical Expenditure Panel Survey (MEPS) produced data indicating that States that expanded Medicaid coverage between late 2013 and late 2014 showed significant reductions in uninsured individuals and emergency room visits. States that did not expand Medicaid coverage over the same time period showed only very slight reductions. A recent blog post for Health Affairs stated that "MEPS data are the number one single source for papers published in Health Affairs."
- AHRQ’s Healthcare Cost and Utilization Project (HCUP) had disseminated more than 50,000 datasets as of October 1, 2016.
Dr. Bindman ended his update by citing AHRQ’s intention to address the concept of a learning health care system. He suggested that the concept will bring depth, focus, and integration to the work of the agency. The National Academy of Medicine developed the following definition: “A learning health care system is one that is designed to generate and apply the best evidence for the collaborative health care choices of each patient and provider, to drive the process of discovery as a natural outgrowth of patient care, and to ensure innovation, quality, safety, and value in health care.” This concept has developed in response to the emergence of the use of EHRs and the trend toward consolidation of health care practices (large systems that can systematically generate and incorporate evidence). Dr. Bindman stressed that AHRQ can play a constructive role in the movement. It can seek to determine how collected information can be fed back into a system to improve care.
Discussion
J. Sanford Schwartz, M.D., asked whether the latest discussions with congressional committees suggested topics to which the NAC should respond. Dr. Bindman noted that representatives tend to begin discussions by inquiring about AHRQ’s unique role and wondering about potential redundancies. There is a continuing need for the broader health care community to educate Congress on AHRQ’s role.
Lucy A. Savitz, Ph.D., M.B.A., asked about the role of the EPCs, especially in light of PCORI’s interest in evidence reviews. Dr. Bindman responded that, because PCORI has indicated an interest in the use of evidence syntheses, he initiated discussions with PCORI leadership about the role AHRQ might play in addressing that need, resulting in the current development of a memorandum of understanding. Ms. Davidson suggested that AHRQ serve to disseminate PCORI information in general. The new SNAC could address that issue. Dr. Schwartz also encouraged AHRQ to play a role in PCORI efforts, perhaps promoting research agendas and making use of the deliberations of the USPSTF.
Naomi Tomoyasu, Ph.D., emphasized the importance of developing each evidence synthesis rapidly. Dr. Bindman agreed, noting that AHRQ is investigating ways to speed up the process.
Dr. Fermazin suggested that AHRQ develop brief descriptions of its unique roles. It could, for example, explain why it develops measures in a way that is complementary to the work of CMS in developing measures.
Referring to the National Quality and Disparities Report, Dr. Ballard proposed that the agency consider describing opportunities for reducing the disparities and using valid measures.
The Learning Health Care System
Comparative Health System Performance: Update and Discussion on Engaging Health Care Systems
David J. Knutson, M.S., Director, Center for Delivery, Organization, and Markets, AHRQ
Mr. Knutson described AHRQ’s program for Comparative Health System Performance (CHSP), which has a broad goal of supporting learning health care systems and performance measurement. The program seeks to identify, classify, and compare health care delivery systems to accelerate the dissemination and implementation of PCOR evidence. The 5-year, $58 million program features three funded Centers of Excellence—Dartmouth/Berkeley, RAND/Penn State, and National Bureau of Economic Research/Health Research and Educational Trust. The three centers offer differences in geographic focus, specific-population focus, and focus on clinical and delivery system interventions.
A specific goal of the program is to produce an AHRQ compendium that will further understanding of the characteristics and practices of high-performing health care systems, leading to additional research and improvement. The compendium will feature a list of all health systems in the United States and provide information on the systems and all providers. The information will include number of facilities by type, size, and composition of medical staff.
Progress to date includes the establishment of workgroups (measures, data, systems definition), initial collections of data and analysis, a coordinating center workshop, and the meeting of a technical expert panel. The participants have agreed on an initial common-denominator definition of health systems for the compendium. They have agreed to harmonize attributes and performance measures and have begun to share ideas about survey contents. Activities to come include the creation of variables, the linking of data and public use files, and the dissemination of data and findings.
Discussion
Ms. Davidson asked whether Medicaid and Medicare data might be used in the program. Mr. Knutson responded that some of that data will be available. Perhaps, added Ms. Davidson, the program could obtain data from the CMS fraud prevention partnership.
Dr. Ballard encouraged the program to consider the complexities of health care systems when performing the analyses. Much of the collected data will not capture complexities. Dr. Weissberg encouraged the program to recognize the fraction of health care work that is handled by physicians in health systems.
Dr. Schwartz suggested implementing a plan to make the data available. Perhaps the program could have interactions with external researchers who might use the data. An advisory board of some kind could facilitate that. Dr. Bindman cautioned that there are some barriers to sharing information. The program will need to determine which data should remain confidential.
Donald A. Goldmann, M.D., raised the problem of the use of terminology. Terms such as “learning system” and “collaborative” can have various interpretations. Learning from the data is a good emphasis for the program; however, a stronger goal would be learning where innovations can succeed. Scaling is important. Dr. Bindman agreed and stated that AHRQ is considering such issues. We need useful ways to extract data to make research less costly and more rapid.
Learning Health Systems: Evidence Implementation and Generation
Arlene S. Bierman, M.D., M.S., Director, Center for Evidence and Practice Improvement, AHRQ
Dr. Bierman described the potential processes for generating and implementing evidence within a learning health care system. The full scope of the learning process proceeds from personal and electronic health records, to information exchange and health analytics, followed by feedback to the public, to practitioners, and to the patient.
Dr. Bierman gave examples of learning communities, including the Medicaid Medical Directors Learning Network, the Innovations Exchange Learning Communities, the EvidenceNOW regional cooperatives, and the EDM Forum program, which partners with AcademyHealth. She provided a hypothetical example of a learning health system that seeks to improve stroke outcomes and reduce costs. Such a system requires evidence implementation at the point of care, making use of guidelines, quality indicators, electronic measures, other tools, and a registry.
The EPCs can play an important role in the process, synthesizing the evidence after it is generated and before it is presented for practice. A national guidelines clearinghouse and a national quality measure clearinghouse are additional tools for the process. AHRQ developed an Integration Playbook to support the integration of behavioral health in primary care. It is Web-based and interactive (www.integrationacademy.ahrq.gov).
Clinical decision support plays a role in a learning system, combining evidence and quality improvement ideas. It must use the right information and intervention format and channel at the right time during the workflow. Patient-reported outcomes/data entered into the EHR represent an additional tool for evidence generation. AHRQ is supporting ongoing projects. The ECHO project is testing the use of tele-health as a factor in a learning health care system.
Dr. Bierman cautioned about special challenges in evidence implementation and generation. Not every provider is tied to a health system. Rural providers often work in isolation. There are financial constraints for small practices. The need for information in primary care is multifarious.
Discussion
Dr. Savitz suggested that it may be beneficial to be proactive by monitoring changes in policy (e.g., new payment penalties). Dr. Naylor proposed that the graphic describing the flow of evidence in the overall system include the role of social sciences. The health care team should be defined broadly. Testing of a learning system should occur across the health care continuum. Kevin L. Grumbach, M.D., added that we should define the health system loosely and differentiate between the system and the science.
Dr. Schwartz applauded the idea of a learning system but cautioned that the dearth of evidence is a serious deficiency. He suggested that AHRQ serve to determine what evidence should be collected. We need rigor and generalizability. We must be careful not to disrupt the functioning of systems.
Dr. Weissberg, referring to the hypothetical example of a project to reduce stroke outcomes, suggested addressing the need to improve systems that connect primary care and subspecialties.
Charles J. Rothwell, M.B.A., M.S., encouraged AHRQ to be involved in developing congressional legislation that relates to policy for evidence-based medicine.
José Escarce, M.D., Ph.D., reminded the group that, after identifying needed behavioral changes and the effects of economics, we need to determine how to create behavioral change. Jennifer E. DeVoe, M.D., D.Phil., M.Phil., MCR, stressed the need for new approaches to evaluate the collected evidence. Dr. Tomoyasu cited a need to consider potential business cases for evidence-based interventions.
Dr. Sherman wondered how we will integrate the collected evidence into delivery, especially with a need to address complex populations. Dr. Bindman raised the issue of support for activities in new infrastructures. We need a workforce that is ready to make the changes.
AHRQ Training Programs: Training LHS Researchers
Francis D. Chesley, Jr., M.D., Director, Office of Extramural Research, Education and Priority Populations, AHRQ
Dr. Chesley reviewed the training and career development programs offered by AHRQ. AHRQ grant training programs span careers, including predoctoral (R36, T32), postdoctoral (F32, T32), and further career development (K08, K01). The agency’s support also features mechanisms under the PCOR Trust Fund. Examples are the PCOR Pathway to Independence Award, Mentored Clinical Investigator Career Development Award in PCOR, and PCOR Institutional Mentored Career Development Program.
The average rate of successful funding for career-development K-award applications is about 36 percent. For R36 awards, the average is about 29 percent. AHRQ conducted an evaluation of the K-award program for grantees receiving funding between 2000 and 2013. Analysis revealed the program to be critical to research career progression and success. Skills improvement showed the greatest gains. Nearly all grantees responding indicated a plan to continue in a research career. A separate analysis of grantees in the dissertation program found that only a small number received subsequent K award funding or independent research grants. About half of the grantees contributed to peer-reviewed publications.
In the year ahead, AHRQ plans to offer an NRSA T32 recompete and a limited competition career development transition-to-independence award (K-R03). The agency is constructing a set of core competencies to guide the development, implementation, and evaluation of training programs for learning health care systems researchers. The move toward a learning health care system requires a conceptual approach for integrating health services research, PCOR, and clinical research. In its next steps, AHRQ will have an expert panel reach consensus on competencies and will develop an institutional training program (K12).
Discussion
Dr. Sherman, referring to the K program evaluation, suggested that AHRQ might be stifling innovation by continually putting forward particular researchers (from one funding to a next funding). Dr. Savitz suggested that AHRQ leverage the application of public use data files in the training programs. Dr. Chesley noted that AHRQ hopes to sponsor a data repository, to use multiple data repositories, and to allow institutions to use their own repositories.
Ms. Davidson proposed a program to bring evidence to medical school curricula. Dr. Goldmann wondered what grant mechanisms will be used by investigators who are embedded. Dr. Chesley stated that a number of mechanisms will be considered. He added that discussions with PCORI have revealed its intention to support such training.
Dr. Escarce asked about the target researchers for the K-award programs. Dr. Chesley responded that AHRQ supports both researchers in health systems and researchers in academics. Dr. Weissberg suggested that the agency’s portfolio include training in patient engagement.
Shari M. Ling, M.D., noted that the effort to develop automatic learning and improvement may create opportunities for focusing on managing populations such as people with complex conditions. Will this be the same as quality improvement efforts? She suggested developing measures other than, for example, publication.
Dr. Schwartz proposed that AHRQ consider funding research in locations other than traditional academic medical centers. Consider regional health centers. Consider delivery systems. We need to include rapid cycle evaluation to understand what works.
Dr. McGlynn stressed a need for decision-makers to provide perspective on their decisions. We need partnerships to help people identify decisions regarding evidence. We must be interactive and include champions as we consider what works and what does not work. There is a tradeoff between perfection and an area in which one can relax. People should receive credit for getting ideas into guidelines or formulary decisions. Let’s revise the incentive system.
Further Discussion with the Panel of Speakers
Dr. Bindman asked the NAC members and speakers to continue to discuss issues regarding development of a learning health care system. What strategies might be used? How can engagement occur? Should we focus on original initiatives or piggyback on current efforts? What steps should be taken? What tools are needed?
Dr. Savitz raised the issue of publication by researchers. Delivery-system researchers need to publish as they compete for external funding. Dr. McGlynn added that embedded researchers should receive credit for work that is funded internally.
Dr. Ballard stressed the importance of system partnerships, training programs, and mentoring. Those aspects can help to bring resources to the table. Ms. Davidson stated that her group created a document on value purchasing, including a “journey map” for the adoption process (including governance, financing, use of technology, and more). That document might be helpful to a learning system program, for example, it could suggest indicators for progress.
Dr. Fermazin cautioned about the multiple interpretations of terminology and the urge to focus on an integrated system. Only 34 percent of physicians work in a health care system. We should begin with individual practices and a focus on communities, seeking to reduce readmissions and to strengthen care coordination. There should be continuous learning in all levels.
Dr. Weissberg asked how the concept of a learning health care system fits with the message of PCORI. Dr. Bindman responded that there is a common observation of vertical systems and the use of best practices based on evidence. Dr. McGlynn cited the promise of outcomes research networks, which identify questions important to patients. Perhaps, added Dr. Bindman, such networks could be expanded with more partners and experiences.
Mr. Knutson stated that the AHRQ comparative health system project is focusing on vertical systems because they offer a feasibility for identifying and linking providers. Dr. McGlynn supported beginning with integrated systems, which offer an informational path forward.
Dr. Naylor cited a need to frame what is to be achieved and perhaps to target populations. We might consider building prototypes for small practices. Dr. Escarce cautioned against the use of secondary data, which are limited in potential. Dr. Bierman added that data from EHRs tend to be unstructured yet have promise. Dr. Escarce suggested studying how high-performing medical institutions are able to address conditions in disadvantaged populations.
Dr. DeVoe encouraged AHRQ to think broadly, considering employers and integrating health insurance claims. Dr. Tomoyasu stated that the Veterans Administration is considering system change and partnerships, which help with implementation and scale-up. Alice S. Bast called for a research summit, featuring stakeholders, to focus on patient needs and the patient point of view. Ms. Davidson noted that vendors now retain vast amounts of data. How might those data be funneled back to health care? Dr. Ling raised the idea of what we do not measure. How can we measure functional limitation?
Dr. Bindman stated that there is no proof of concept for creating a learning health care system. Dr. DeVoe stressed a need for researchers to demonstrate that they have a robust laboratory.
Dr. Weissberg proposed a value in beginning not by studying primary care but by studying a circumscribed medical care practice (e.g., cataract care). We could proceed from there. Dr. Bierman added that it may be necessary to improve our capacity for measurement at the start. Dr. Naylor proposed the use of exemplars to establish pillars for a learning health care system. Dr. Ballard proposed beginning with a focus on disparity reduction.
Mr. Rothwell argued for addressing primary care physicians, and he cautioned against conducting research for the sake of research. Dr. McGlynn proposed pre-analytic work to learn what we want to know and to test change. We need to learn about the process of learning. Many methods are reusable. Another possible focus for initial efforts might be patients with multiple chronic conditions.
Dr. Tomoyasu called for defining the core characteristics of a particular learning health care system. Dr. Bindman called for the development of indicators of progress. He asked the NAC members to propose the appropriate partners (health systems) for this work.
Dr. McGlynn stated that we need to determine how to create a synergy that keeps people learning over long timespans. We need mixed methods and respect for the context. Dr. Weissberg cited work in developing small practice improvement cycles. Dr. Naylor stressed that the frontline clinical team must be included when seeking input from stakeholders (also board members in health systems).
Public Comment
Lindsey Horan, M.A., of AcademyHealth, reminded the NAC members that her independent group, Friends of AHRQ, continues to support the agency. Its "Save AHRQ Campaign" has achieved traction in the health care community, bringing attention to the agency. The group continues to work with congressional representatives to advance AHRQ’s interests. It continues to seek and engage AHRQ champions on Capitol Hill. Ms. Horan has witnessed an enthusiasm for the agency there. She asked the NAC members to consider getting involved.
Adjournment
Dr. McGlynn asked the council members to forward any ideas for the agendas of future meetings. Dr. Bindman suggested that one topic could be measuring the impact of AHRQ. The next NAC meeting will take place on March 24, 2017, in the Hubert H. Humphrey Building in Washington, DC. Drs. McGlynn and Bindman thanked the NAC members, speakers, and other guests, and adjourned the meeting.
Respectfully submitted,
Elizabeth A. McGlynn, Ph.D., Chair
National Advisory Council
Agency for Healthcare Research and Quality
