Recommended Data Collection Procedures for CAHPS Surveys

Each survey sponsor will need to work with their survey vendor to choose the data collection modes that are most likely to reach their specific patient population and maximize the response rate at an acceptable cost. Research suggests that sequential multi-mode approaches are more effective than using a single mode. Organizations often want to use the least expensive mode first in multi-mode protocols (such as web before mail, mail before phone, etc.), but the specific combination/sequence used will depend on the organization's capabilities, available contact information, and the most recent research regarding different survey modes. Described below is an example of one possible sequence: web-mail-phone.

There is no specific cut-off point for the field period and duration of data collection will vary depending on the mode or modes of data collection being used. For example, a survey administered using a mail-phone protocol may have a field period of 10 to14 weeks, while a survey administered using a web-mail protocol may have an 8-week field period. Data collection should not stop just because the target number of completed questionnaires is reached. The data collection protocol should be completed as planned to ensure comparability with the results of other surveys that have a comparable field period and to achieve the highest possible response rate. You may need to adjust your field period based on the population you are surveying and the response rate you are getting.

First Steps: Preparing for Data Collection

There are a few steps to take before beginning the data collection process:

• Verify email addresses. Check the email addresses of sampled patients to identify and remove email addresses that don’t conform to the format of a prefix, an “@” symbol, and a domain name (such as example@email.com), as those addresses are not likely to be valid. Some survey vendors subscribe to email verification tools that can test whether each email in your sample exists and is able to receive messages. This step is optional as removing emails that don’t conform to the valid email format will remove most, if not all, “undeliverable” email addresses.
• Update and format addresses. Use a national change of address (NCOA) service to identify and update addresses for sampled patients using US Postal Service NCOA data and format each patient address to conform to UPS Postal Service standards, which promotes timely, accurate delivery. Such services can also provide Coding Accuracy Support System (CASS) processing to identify and flag addresses that are unable to receive mail.
• Verify phone numbers. Check the phone numbers of sampled patients for out-of-date area codes and partial or unlikely phone numbers. Survey vendors should have standard automated procedures for checking and updating phone numbers before beginning data collection.
• Set up a toll-free phone number that patients can use to ask questions. Some organizations also provide an email address. Include this contact information in all correspondence with sampled patients. Assign a trained project staff member to respond to questions that are submitted. It is useful to maintain a log of these calls and emails and review them periodically.
• Prior to sending the survey, send a letter describing the survey and notifying the recipient that they will receive a survey. This step is recommended because it can help to increase responses, identify bad addresses, and identify deceased patients. The letter may include instructions to complete the survey via the web with a URL address. Frequently, sponsors use a scannable Quick Response (QR) code to facilitate easy access for respondents to complete the web survey.

Learn more: Preparing Sample Files Before Collecting CAHPS Survey Data.

Web Mode

In web mode, respondents use a web browser to read and answer the survey questions on a screen. Invitations to a web survey can be sent in multiple ways, including email, text messages, QR codes, patient portals, and mailed letters. Research has shown that email invitations result in higher response rates.

Below are some basic steps for contacting sampled patients via email to invite them to take a web-based survey and make this process as effective as possible.

• Send the patient an invitation with a link to the online survey. A well-written, persuasive message authored by a recognizable organization will increase the likelihood that the recipient of the survey invitation will complete it within the deadline. The invitation should be personalized and contain a direct link to the survey. This link should consist of the survey URL and an ID or PIN code that is unique to the sampled individual. When the recipient clicks on the link, they should go directly into their personal survey. The invitation should also explain whom to contact if recipients have questions.

• Send a reminder email to nonrespondents 5 to 7 days after sending the initial invitation.
• Send a second reminder to those still not responding 5 to 7 days after the first reminder.
• Follow up with nonrespondents by mail or phone. It is critical to initiate contact by either mail or phone with those who have not completed the survey online. Since not all patients have access to or use email regularly, survey sponsors must follow the web mode with the mail and/or phone mode for all nonrespondents to ensure that the final survey responses represent the patient population that was sampled.

Mail Mode

In a mail mode, surveys are sent to sampled patients via the mail (first class or priority mailings). Below are some typical basic steps for collecting data through the mail and advice for making this process as effective as possible.

• Send the printed survey with a cover letter and a postage-paid return envelope to sampled patients (or nonrespondents if this is follow-up to the web mode). A well-written, persuasive letter from a recognizable organization will increase the likelihood of a timely return. The cover letter should include instructions for completing and returning the survey.

• Consider sending a postcard reminder to nonrespondents 10 days after sending the questionnaire. Some vendors prefer sending a reminder postcard to all sampled patients 3 to 5 days after mailing the questionnaire instead of sending a postcard to only nonrespondents 10 days after the questionnaire is mailed. Their reminder postcards serve as a thank you to those who have returned their questionnaires and as a reminder to those who have not.
• Send a second mailing with a reminder letter, printed survey, and a postage-paid return envelope to those still not responding 3 weeks after the first mailing.

Phone Mode

When collecting data by phone, surveys must be modified for phone administration. The phone interview will need an introductory script that communicates the information contained in a mail survey cover letter and additional scripting to ensure that respondents hear all response choices for survey questions.

• Train the interviewers before they begin interviewing. The interviewer should not bias survey responses or affect the survey results.
• Phone nonrespondents. If following up on an email or a mailed questionnaire, initiate phone contact with nonrespondents 3 weeks after sending the second reminder. You may want to send a letter to sampled patients in advance to let them know that you will be contacting them by phone.
• Attempt to contact each nonrespondent by phone at least six times. The vendor should make at least six attempts unless the sampled patient explicitly refuses to complete the survey. These attempts should be on different days of the week (both weekdays and weekends), at different times of the day, and in different weeks.