Development of the CAHPS Clinician & Group Narrative Item Set

The first CAHPS Narrative Item Set was designed to enable users of the CAHPS Clinician & Group Survey to elicit a short, coherent story from patients about their experiences with care. These items were originally referred to as the Patient Narrative Elicitation Protocol. They were developed and tested by researchers affiliated with the Yale School of Public Health, RAND, and the University of Wisconsin-Madison.

Development of the Initial Protocol

Building on previous patient narratives research, the team began by designing a sequence of questions applicable to and readily understood by patients with more or less complex health care experiences, as well as those with various levels of health literacy. To confirm that the questions were understandable and clear, the team conducted one-on-one interviews with a diverse group of health care consumers. Initial versions of the questions were then tested with roughly 30 respondents, and wording was revised accordingly.

Experimental Testing of the Protocol

To assess the performance of the initial item set, the team conducted an experimental test by administering the CAHPS Clinician & Group Adult Survey in English with the short set of open-ended questions to a sample of 443 respondents using both telephone and web-based response modes. Survey respondents were invited to answer the open-ended questions after completing the survey. To evaluate the narrative information collected by the survey questions, the team compared the content to information obtained during hour-long interviews with a subsample of 50 survey respondents who were interviewed 2 to 3 weeks later. By carefully coding all of the content, the research team was able to assess the degree to which the information in the open-ended survey responses matched the "real story" obtained through the interviews.

Based on the results of this first experiment, the team revised the Narrative Elicitation Protocol. To improve the flow of the narrative, the team restructured the questions into a more "storytelling" mode, starting with a question on what patients expect or seek from their health care providers. To promote more detail, particularly in the written responses, additional prompts to the third and fourth questions were added in the sequence. Finally, to remind respondents to comment on the nature of their relationships with clinicians (and staff) as well as the events that comprised their medical experiences, the team revised the fifth question to focus explicitly on how patients interact with their provider.

In a second round of experimental testing with 332 respondents, the team assessed responses to these revised items. As in the first round, a subsample of 50 respondents participated in intensive interview by phone. The changes to the item set resulted in an increase in the completeness of elicitations from roughly 40 percent to approximately 60 percent of the content gleaned from the hour-long interviews.

More information on the development of this Narrative Elicitation Protocol is available in Grob R, Schlesinger M, Parker AM, et al. Breaking Narrative Ground: Innovative Methods for Rigorously Eliciting and Assessing Patient Narratives. Health Serv Res 2016 April; 51:1475-6773.

Field Testing of the Protocol

To assess how well the Narrative Elicitation Protocol performs outside of the restrictions of an experiment and better understand how the responses can be used, the CAHPS team field-tested the protocol in a pilot project conducted by the California Healthcare Performance Information System (CHPI) and Massachusetts Health Quality Partners (MHQP). This test found that:

• 18–25% of respondents agreed to answer open-ended responses in a scenario in which they had to actively opt-in to do so after the closed-ended survey questions.
• The additional questions did not exhaust respondents’ enthusiasm for participating in the survey. Virtually all of those respondents completed the full set of five items; response rates for those items declined only 0.5 percentage points from the first question to the fifth.
• The five-question protocol performed substantially better at capturing patient experiences than a shorter, alternative sequence of three open-ended questions.

Release of a Beta Version

AHRQ released the beta version of the CAHPS Narrative Elicitation Protocol in early 2017. The CAHPS team continued to investigate ways to refine the questions, strategies for encouraging survey participation, and methods for most effectively analyzing and reporting the information obtained through comments to make them useful to clinicians and other patients.

Release of the Final Version of the CG-CAHPS Narrative Item Set

Based on additional testing of the Narrative Items for adults and children in English and Spanish, the CAHPS team refined the CG-CAHPS Narrative Item Set and released it in June 2021. There are two versions of this Narrative Item Set:

• Items for adults and children in English and Spanish designed for the CAHPS Clinician & Group Survey 3.0. 
• Items for adults in English only designed for the CAHPS Clinician & Group Visit Survey 4.0 (beta).

Research Articles Published by the CAHPS Team

• Grob R, Schlesinger M, Parker AM, et al. Breaking Narrative Ground: Innovative Methods for Rigorously Eliciting and Assessing Patient Narratives. Health Serv Res 2016 Jun;51:Suppl 2:1248-72.
• Martino SC, Grob R, Davis S, et al. Choosing Doctors Wisely: Can Assisted Choice Enhance Patients’ Selection of Clinicians? Med Care Res Rev 2017 Nov 25. Epub ahead of print.
• Martino SC, Shaller D, Schlesinger M, et al. CAHPS and Comments: How Closed-Ended Survey Questions and Narrative Accounts Interact in the Assessment of Patient Experience. J Patient Exp 2017 Mar;4(1):37-45.
• Kanouse DE, Schlesinger, M, Shaller D, et al. How patient comments affect consumers’ use of physician performance measures. Med Care 2016 Jan;54(1):24-31.
• Schlesinger M, Grob R, Shaller D. Using Patient-Reported Information to Improve Clinical Practice. Health Serv Res 2015 Dec;50(S2):2116-2154.
• Schlesinger M, Grob R, Shaller D, et al. Taking Patients’ Narratives about Clinicians from Anecdote to Science. N Engl J Med 2015 August;373(7):675-679.
• Schlesinger M, Kanouse DE, Martino SC, et al. Complexity, public reporting, and choice of doctors: a look inside the blackest box of consumer behavior. Med Care Res Rev 2014;71:Suppl:38S-64S.
• Schlesinger M, Kanouse DE, Rybowski L, et al. Consumer response to patient experience measures in complex information environments. Med Care 2012;50:S56-S64.