Methods for Increasing the Number of Responses to CAHPS Surveys

Many organizations have reported a decline in survey response rates. A low response rate is a concern because a survey with a low response rate may provide data that is less representative of the underlying population, although that is not always the case. 

Differences in response rates across units of interest (individual providers, practices, or medical groups) are also a concern, because this may be related to differential bias across units. Survey users should track the individual and unit-level response rates for all surveys. 

Inaccurate mailing and email addresses, inaccurate phone numbers, voicemail, gatekeepers, and frequent travel by respondents are common problems when trying to contact potential respondents. To maximize the number of responses, sponsors and vendors can: 

• Improve initial contact rates by making sure that addresses, phone numbers, and email addresses are current and accurate (e.g., identify sources of up-to-date sample information, run a sample file through a national change-of-address-database, send a sample to a phone number lookup vendor). 
• Take steps to improve contact rates after data collection has begun (e.g., make multiple calls, make calls at different day and evening times over a period of days, mail second reminders, and use experienced and well-trained interviewers). 
• Consider using a mixed-mode protocol involving email, mail, and phone data collection procedures. Multi-mode approaches tend to achieve a higher response rate than a single mode. 
• Train interviewers on how to deal with gatekeepers (persons other than the target respondent). 
• Train interviewers on refusal aversion/conversion techniques. 

These methods will add to the costs of conducting a survey, but users need to weigh these extra costs against the risk of obtaining low response rates and less representative data. 

Once the vendor reaches the potential respondent, there are other challenges: people throw away the envelope, sometimes unopened, or set aside the questionnaire but then never complete it. These responses draw attention to the importance of effectively communicating why the person should complete the questionnaire. In addition to persistent follow-up, make sure that the outside envelope, cover letter, and questionnaire are as attractive and compelling as possible. 

It is especially important to interview or receive returned questionnaires from those individuals who might be difficult to reach. They are likely to be different from those individuals who immediately complete and return a questionnaire or who are easily interviewed. They may, for example, be chronically ill, have two jobs, or be different in some other way that is relevant to your results. Unless you maintain a high response rate overall and make efforts to reach them, their views and experiences will be underrepresented. 

Sponsors and vendors should discuss this possibility in advance and consider plans to do extensive phone tracking and locating. You may also want to talk about the timing of interviews. Surveys of respondents at their homes require interviewers to typically work in the evenings and on weekends. However, the survey vendor should provide at least one interviewer during the daytime to maintain appointments made with respondents during the day and try to reach those respondents who do not answer during the evenings (e.g., those who have evening shift jobs). Interviewing during the daytime on weekdays is especially effective and appropriate for surveys of seniors and surveys that include children in the sample frame.

You are likely to encounter certain types of problems with which you should be familiar. Sponsors and vendors should discuss these issues and agree on appropriate procedures.