Development of the CAHPS American Indian Survey

The development of the CAHPS American Indian Survey followed the standard CAHPS process.

Meeting with the Choctaw Nation Health Services (CNHS). In February 2004, CAHPS team members from RAND and AHRQ met with CNHS staff members in order to learn about CNHS's information needs, organizational structure, care delivery system, and past experiences with patient surveys. This meeting also helped the team identify and prioritize concrete objectives for the study.

Development of a first draft. CAHPS team members led by RAND developed a list of preliminary domains and survey measures for CNHS staff, who provided additional feedback and identified potential candidates for inclusion in the instrument. The CAHPS team proceeded to develop a first survey draft with 65 items and 19 domains. That draft built on the Consortium's parallel work to develop the CAHPS Clinician & Group Survey.

Cognitive testing. In the summer of 2004, the RAND members of the CAHPS team conducted cognitive testing of the first draft of the CAHPS American Indian Survey with 20 subjects recruited from 3 CNHS outpatient clinics. These tests indicated a generally high level of relevance among the survey items and domains, and few problems in terms of respondent comprehension. They also revealed a need for some revision to clarify service structures in specific clinics and to allow CNHS to gather clinic-specific information.

Field testing. In the summer of 2005, RAND fielded a revised draft instrument, with 81 items and 16 domains. RAND, AHRQ, and CNHS worked together to develop specifications for drawing the sample, which included 1200 randomly selected adults who had visited any of five CNHS outpatient clinics. The administration mode was two-wave mail only, which achieved an overall response rate of 58%.

An analysis of the field test results indicated that the draft instrument by and large demonstrated a high level of reliability and measured its intended domains well. Based on their findings, the CAHPS team made only a few modifications to the questionnaire, namely minimizing the perceived discrimination composite from 6 to 2 items, and removing one item from the health education composite.

Read a summary of the Field Test Report. (PDF, 103 KB)

The development process for this survey has been presented at the following national conferences:

• Weidmer B, Johanson P, Dalpoas D, Wharton D, Hays RD, Darby C. Adapting CAHPS^® for an American Indian Population: Choctaw Nation Health Services Pilot Project. Paper presented at the National Indian Health Board, 21st Annual Consumer Conference, August 2004, Oklahoma City, OK.
• Hays RD, Johanson P, Weidmer B, Wharton D, Dalpoas D, Darby C. Psychometric assessment of an experience with care survey for American Indians. Paper presented at the Society for General Internal Medicine annual meeting, April 2006, Los Angeles, CA.