Development of the CAHPS Adult Hospital Survey

The CAHPS Hospital Survey was conceived first and foremost as a tool for public reporting purposes. While many hospitals already conduct some type of patient survey, CMS needed a standardized instrument that would allow consumers to compare inpatients' perspectives on the quality of care fairly and reliably across facilities. It also wanted an instrument that met high standards for scientific rigor and salience with consumers. AHRQ’s CAHPS program provided an ideal setting for the development of such an instrument, and after preliminary collaboration in early 2002, the idea for the CAHPS Hospital Survey was born. To ensure that the perspectives and concerns of hospitals were well understood and addressed during the development of this instrument, CMS and AHRQ engaged with the Hospital Quality Alliance, in a public-private collaboration to foster greater accountability and improvements in hospital care.

The initial input for measures. In July of 2002, the Agency for Healthcare Research and Quality (AHRQ) published a "Call for Measures" in the Federal Register that asked organizations to suggest survey items for the CAHPS development team to consider. Seven organizations responded by providing instruments and items for consideration.

The CAHPS team also drew on other sources, including:

• Items from the CAHPS Health Plan Survey for adults.
• Web chat questions and comments (October 24, 2002).
• Stakeholders' meeting (November 7, 2002).
• Vendors' meeting (November 18, 2002).
• Responses to an electronic mailbox for the CAHPS Hospital Survey.
• Literature review conducted by the RAND CAHPS Team.
• Results of cognitive testing of the instrument (below).

OMB clearance of the initial draft. Using the input received from these sources, the CAHPS Team developed a draft of the Hospital Survey instrument and submitted it to CMS on January 15, 2003. CMS then submitted this draft to the U.S. Office of Management and Budget (OMB) for clearance.

As part of the OMB clearance process, CMS published a Federal Register notice on June 27, 2003, asking for comments on the draft instrument as well as input about implementation options. CMS and AHRQ responded to these comments in accordance with OMB's Paperwork Reduction Act requirements. Four additional notices were published in the Federal Register, providing ample opportunity for public input. The fifth and final notice was published in the summer of 2005.

Pilot testing. CMS conducted pilot tests of the draft instrument through a contract with the Quality Improvement Organizations (QIOs) in three states: Arizona, Maryland, and New York. This pilot test included 132 hospitals and resulted in over 19,000 completed surveys. Testing began in June 2003 and ended in August 2003. Read the HCAHPS Three-State Pilot Study Analysis Results.

Focus groups. AHRQ and CMS conducted six focus groups with consumers in October 2003 and another 10 in March 2004. These focus groups were conducted in four cities and included adults who had recently been in a hospital or who were caregivers for someone who had recently been in a hospital.

Additional field testing. Over a 6-month period beginning in the fall of 2003, the CAHPS Team tested the instrument in five volunteer sites encompassing over 375 hospitals:

• Calgary Health Region.
• California Institute for Health System Performance.
• California Regions of Kaiser Permanente.
• Massachusetts General Hospital.
• Premier Incorporated.

The CAHPS Team used these field tests to learn more about the instrument, sampling processes, data collection processes, and other issues related to survey implementation.

Pre-implementation testing. In the summer of 2004, the CAHPS Team worked with hospitals and vendors that volunteered to test the current instrument in order to identify ways to minimize the potential burden and disruption posed by this survey. In collaboration with these test sites, researchers investigated various approaches to integrating the survey items into existing questionnaires as well as alternative protocols for administering the survey.

More pre-implementation testing took place in the spring and summer of 2005. Participants used this opportunity to integrate their own items into the CAHPS Hospital Survey and experiment with other issues related to sampling and survey administration.

Endorsement by NQF. In the fall of 2004, having concluded the instrument testing process, AHRQ and the CAHPS Team gave CMS their recommendations for the questionnaire and administration guidelines. CMS submitted a 25-item instrument to the National Quality Forum's (NQF) review and consensus-building process for endorsement. An NQF committee made some recommendations, including a request to add back two items, to which CMS responded. After a period of public review and comment on the 27-item version, the NQF membership endorsed the CAHPS Hospital Survey as a measure in May 2005.