Formatting CAHPS Questionnaires for Mail, Web, and Phone Administrations

To ensure consistency across surveys, please observe the following guidelines when formatting your survey to be administered by mail, web, and/or phone. Survey users should carefully consider which mode(s) would be the most desirable and appropriate for their patient population. Users who plan to conduct sequential mixed-mode surveys combining web and mail visual modes should strive to design the surveys to look similar.

Formatting the Questionnaire for Mail Surveys

The CAHPS team strongly recommends taking steps to maximize the visual appeal and readability of mailed questionnaires. A well-formatted questionnaire contributes to response rates, completion rates, and the accuracy of responses.

The surveys available from the AHRQ website are formatted to be self-administered mail questionnaires that are designed to be easy to read. Important elements include the two-column format, the use of white space, and the font size and type (12 point Times New Roman). Please retain these elements if you add supplemental items to your questionnaire. The table below provides detailed recommendations for formatting the survey.

To minimize the number of pages required to print the questionnaire, survey users are often tempted to orient response options horizontally instead of vertically, especially for the 0-10 ratings items. However, research has shown that respondents are often confused by the horizontal orientation, which can lead to invalid responses. Therefore, the CAHPS team strongly recommends maintaining the vertical orientation that is shown in the formatted questionnaires.

Table. Guidelines for Questionnaire Formatting

Formatting the Front Cover

The front cover of the formatted questionnaires is designed to identify the questionnaire for survey users and vendors; it is not intended for distribution to patients. The CAHPS team recommends developing a customized cover that explains the purpose of the survey and identifies its sponsor. Be sure to include:

• The name of the survey sponsor.
• The sponsor’s logo, if appropriate.
• A statement about the voluntary nature of the survey and the confidentiality of all responses. Please use or adapt the statement provided with the questionnaires.

Respondents may not recognize the names of some survey sponsors, such as community alliances or collaboratives. In those cases, consider including the name of an entity that is more familiar to respondents, such as the provider’s practice. This tactic increases the likelihood that the respondent will complete the survey.

Formatting Web-Based Surveys

Web-based surveys can offer convenience to respondents and cost-effectiveness to survey users. However, it is important to keep in mind that not all patients are equally likely or able to respond to a web survey. Survey users need to consider whether the web is an appropriate mode for their patient population.

If you determine that a web-based survey is appropriate for you, please consider the following guidelines:

• Follow the general formatting guidelines provided for the print version (with respect to spacing, vertical response options, etc.). If using both mail and web versions of the survey, keep the survey formats as consistent as possible to reduce the potential for differences between modes due to formatting. 
• Keep the screen design simple so that the survey experience is similar across a range of internet browsers, e.g., Chrome, Firefox, Safari, and platforms, including personal computers, Macs, tablets, and smart phones.
• Do not use advanced web survey tools such as graphic sliders for 0 to 10 scales or matrix tables to present multiple questions using the same response options. Keep the look and feel of the mail survey design in the web survey.
• Work with your vendor or in-house survey team to test the survey on multiple browsers and platforms, including smart phones. When testing the survey in different browsers and platforms, check for consistency in:
  • The presentation of images, colors, and fonts.
  • Functionality (i.e., all features operate as intended).
  • Usability, particularly on mobile devices.
• Avoid large logos or design elements at the top of the page as this increases the need for scrolling.
• Use a font type that promotes readability on a web page.
• Provide a welcome screen that emphasizes the ease of response, is motivational, and instructs the user on how to proceed to the survey.
• Use a single column and display all questions in a section or topic on one screen to complete the survey on mobile devices.

Formatting for Telephone Surveys

If you are planning to implement telephone surveys, please keep the following guidelines in mind:

• If you include CAHPS supplemental items in your survey, follow the placement instructions provided with the items.
• If you add questions to the survey, adapt them to the format used in your script. For instance, for all questions that use the "How Often" response scale, the interviewer should say "Would you say…" before reading the response options to the respondent.
• Include a "DON’T KNOW" (DK) and "REFUSED" (REF) response option in the Computer Assisted Telephone Interview (CATI) program or other data collection tool. "DON'T KNOW" and "REFUSED" responses would typically not be read aloud on the telephone. Unless otherwise noted, "DON'T KNOW" and "REFUSED" responses should follow the same skip pattern as the "NO" response option.
• Omit the last two questions of the core survey that ask about receiving assistance in completing the questionnaire. These are not included in telephone scripts because interviews should not be conducted with proxy respondents.