Aging and Disability Resource Centers
Tool or Measure Set: Aging and Disability Resource Centers (ADRCs): Selected Measures of Streamlining
Data Source: Administrative data
Respondent: N/A
Mode of Data Collection: N/A
Total Items/indicators: 54 measures
HCBS MS Domains: Effectiveness, Person Centeredness/Autonomy, Timeliness, Efficiency, Access, and Unmet Need
Populations: Elders with physical or cognitive disabilities, adults or children with mental retardation or developmental disabilities, children with physical, cognitive, or mental health disabilities, adults with severe and persistent mental illness, adults with acquired brain injuries, and adults with physical disabilities and/or chronic conditions
Developer: ADRC Technical Assistance Exchange/The Lewin Group
Use history: Measures document posted May 23, 2006.
States Currently Using: Used by ADRCs nationwide
Psychometric Testing: Not specified
Summary: Measures created to provide consistent measurement across Aging and Disability Resource Center grants funded through the Administration on Aging and the Centers for Medicare and Medicaid Services. Focus areas include initial calls, basic information, comprehensive needs assessment, providing assistance, eligibility determination, and ongoing monitoring.
Tool or Measure Set: Aging and Disability Resource Centers (ADRCs): Consumer Satisfaction Survey Questions by Domain
Data Source: Consumer Survey
Respondent: Consumer or Proxy
Mode of Data Collection: May differ by State
Total Items/indicators: Collection of all surveys results in a total of 77 items and 19 additional demographic questions
HCBS MS Domains: Effectiveness, Person Centeredness/Autonomy, Timeliness, Efficiency, Access, and Unmet Need
Populations: Elders and individuals with disabilities
Developer: ADRC Technical Assistance Exchange/The Lewin Group
Use history: Document posted on June 24, 2004
States Currently Using: Developed for use by ADRCs nationwide
Psychometric Testing: Not specified
Summary: The Technical Assistance Exchange reviewed 12 existing consumer satisfaction surveys used at the local, State, and national program level and produced a comprehensive list of relevant items as well as a matrix highlighting covered content areas. Matrix also provides information on survey development and implementation. Document provides a comprehensive list of consumer satisfaction survey questions used across the Aging and Disability Center grantees. Focus areas include reliability, comprehensiveness, currency and usefulness of information, simplicity and clarity, staff interaction, preferences and unique circumstances, courteousness, communication, and overall satisfaction.
Tool or Measure Set: Aging and Disability Resource Centers (ADRCs): Recommended Consumer Measures for Surveys
Data Source: Consumer Survey
Respondent: Consumer or Proxy
Mode of Data Collection: Not specified
Total Items/indicators: 16 close-ended items, 1 open-ended item, and 9 demographic questions
HCBS MS Domains: Effectiveness, Person Centeredness/Autonomy, Timeliness, Efficiency, Access, and Unmet Need
Populations: Elders and individuals with disabilities
Developer: ADRC Technical Assistance Exchange/The Lewin Group
Use history: Document dated June 24, 2004
States Currently Using: Developed for use by ADRCs nationwide
Psychometric Testing: Not specified
Summary: A draft satisfaction survey was created for States to modify and adopt for use in their Aging and Disability Resource Networks. Focus areas include customer services, application for services, and general experiences.
Return to Appendix III Contents
Centers for Medicare & Medicaid Services
Tool or Measure Set: The Guide to Quality Measures: A Compendium, Medicaid and SCHIP [State Children's Health Insurance Program] Quality Improvement, Version 1. 0
Data Source: Multiple: administrative, consumer survey, chart review, medical record
Respondent: N/A
Mode of Data Collection: N/A
Total Items/indicators: 351 measures
HCBS MS Domains: Health and Welfare, Safety, Effectiveness, Access
Populations: Medicaid recipients
Developer: Compiled by the Division of Quality, Evaluation, and Health Outcomes, Family and Children's Health Programs Group, Center for Medicaid and State Operations, Centers for Medicare & Medicaid Services (CMS)
Use history: Published in September 2006
States Currently Using: Resource for States nationwide
Psychometric Testing: Measures drawn from existing measure sets, many of which have established reliability and validity
Summary: CMS encourages the use of nationally recognized, vetted, and tested measures in State quality improvement activities, when possible. This list contains quality measures in broad categories to support States' programmatic needs in a variety of quality areas, including pay for performance, public reporting, and quality improvement. These established measures are drawn from a variety of sources, including the Joint Commission for the Accreditation of Healthcare Organizations (JCAHO, now known as the Joint Commission) National Committee for Quality Assurance (NCQA), and CMS. The compendium provides a description of each measure, the relevant setting and population, the source and type (e.g., outcomes, process) the data source, and whether it has been endorsed by the National Quality Forum. Measures are primarily condition specific; many could be applied to HCBS population.
Tool or Measure Set: Participant Experience Survey, Elderly and Disabled Version (PES E/D)
Data Source: Consumer survey
Respondent: Consumer or Proxy
Mode of Data Collection: Face-to-face structured interview
Total Items/indicators: 33 indicators
HCBS MS Domains: Access, Unmet Need, Person Centeredness/Autonomy, Health and Welfare
Populations: Elders with physical disabilities, elders with cognitive impairments, adults with physical disabilities and chronic illnesses
Developer: The Medstat Group, Inc., under contract to CMS
Use history: Version 1. 0 released in August, 2003. Survey tool and supporting materials are in the public domain.
States Currently Using: At least 12 States nationwide
Psychometric Testing: Development and testing included cognitive testing, field testing, and limited reliability and validity testing. Stakeholder involvement to insure content validity
Summary: The PES E/D is a 67-item consumer survey developed under contract to CMS and designed to solicit direct feedback from participants about their experience with HCBS waiver services. The resulting indicators can be used to guide quality improvement efforts. Approximately one dozen States have used, are using, or are preparing to use this tool. Free automated survey software is available for both data collection and analysis.
Tool or Measure Set: Participant Experience Survey, MRDD [Mental Retardation/Developmental Disabilities] Version (PES MRDD)
Data Source: Consumer survey
Respondent: Consumer or Proxy
Mode of Data Collection: Face-to-face structured interview
Total Items/indicators: 51 indicators
HCBS MS Domains: Access, Unmet Need, Person Centeredness/Autonomy, Health and Welfare
Populations: Adults with intellectual disabilities
Developer: The Medstat Group, Inc., under contract to CMS
Use history: Version 1. 0 released in August, 2003. Survey tool and supporting materials are in the public domain.
States Currently Using: Approximately 3-5 States nationwide
Psychometric Testing: Cognitive and field testing, stakeholder involvement to insure content validity
Summary: The PES MRDD is a 97-item consumer survey developed under contract to CMS and designed to solicit direct feedback from participants about their experience with HCBS waiver services. The resulting indicators can be used to guide quality improvement efforts. Approximately five States have used, are using, or are preparing to use this tool. Development and testing included cognitive testing, and field testing. Free automated survey software is available for both data collection and analysis.
Tool or Measure Set: Participant Experience Survey, Acquired Brain Injury Version (PES BI)
Data Source: Consumer survey
Respondent: Consumer or Proxy
Mode of Data Collection: Face-to-face structured interview
Total Items/indicators: Up to 58 indicators, depending on the number of survey modules used
HCBS MS Domains: Access, Unmet Need, Person Centeredness/Autonomy, Health and Welfare
Populations: Adults with acquired brain injuries
Developer: The Medstat Group, Inc., under contract to CMS
Use history: Version 1. 0 released on May 10, 2005. Survey tool and supporting materials are in the public domain.
States Currently Using: Limited use history
Psychometric Testing: Cognitive and field testing, stakeholder involvement to insure content validity
Summary: The PES BI was developed under the guidance of an advisory group of administrators from State Medicaid HCBS programs for persons with acquired brain injuries. Items subjected to several rounds of cognitive testing, plus one large-scale field test. The survey was designed to solicit direct feedback from participants about their experience with HCBS waiver services. The resulting indicators can be used to guide quality improvement efforts. There are 52 items that apply to all respondents, and two optional modules-additional community integration and activities of daily living/instrumental activities of daily living (ADL/IADL) questions-that can be used depending on the service scope of a State's brain injury waiver program. Currently the survey is used by a small number of States for quality management purposes.
Tool or Measure Set: Quality Measures Management Information System (QMIS)
Data Source: Multiple-measures are drawn for existing tools and measure sets.
Respondent: N/A
Mode of Data Collection: N/A
Total Items/indicators: There are more than 300 measures in the database; 27 have been deemed relevant to home health.
HCBS MS Domains: Unmet Need, Health and Welfare
Populations: Medicare and Medicaid beneficiaries
Developer: The Centers for Medicare and Medicaid Services
Use history: Site is open to the public and maintained by the Iowa Foundation for Medical Care
States Currently Using: Unknown
Psychometric Testing: Summaries of individual measures include information about endorsements, but not testing.
Summary: The Quality Measures Management Information System (QMIS) is a comprehensive, Web-based, electronic tool to support the Measures Management System. It is designed to be the repository of all the quality measures used by CMS and the electronic tool to track the development and maintenance of those measures. This online tool can be used to locate technical specifications, justification or the history of a particular measure. Searches can be conducted by developer, care setting, quality domain, measure type, CMS use, and body system. The home health measures are primarily Outcome and Assessment Information Set (OASIS) measures or measures of appropriate care for people with end-stage renal disease.
Tool or Measure Set: Outcome and Assessment Information Set Home Health Quality Measures
Data Source: Assessment data
Respondent: N/A
Mode of Data Collection: Assessment data are submitted and collated electronically
Total Items/indicators: 41 measures: 30 are Risk-adjusted quality measures and 11 are Descriptive quality measures
HCBS MS Domains: Unmet Need
Populations: Elders and nonelders receiving federally funded home health services from Medicare-certified home health agencies
Developer: Center for Health Services Research at the University of Colorado.
Use history: Medicare-certified home health agencies have been required to submit OASIS data since 2000. Ten quality measures calculated from these data are published through the Home Health Compare project.
States Currently Using: Used by home health agencies nationwide
Psychometric Testing: Reliability and validity have been examined in peer-review articles.
Summary: Patient outcome measures are calculated on a completed episode of care, which begins with admission to a home health agency (or a resumption of care following an inpatient facility stay) and ends with discharge or transfer to inpatient facility. Publicly reported measures were endorsed by the National Quality Forum in 2005. Measures primarily assess improvements or stabilization in physical and cognitive functioning following postacute home health care, and may not be relevant to HCBS.
Tool or Measure Set: Quality Indicators for Medicaid Services to People with Developmental Disabilities
Data Source: Multiple
Respondent: N/A
Mode of Data Collection: N/A
Total Items/indicators: 33 indicators
HCBS MS Domains: Safety, Effectiveness, Person Centeredness/Autonomy, Equity, Access, Qualified Providers, Health and Welfare, Administrative Oversight, Unmet Need, and Quality of Life
Populations: Adults and children with mental retardation or developmental disabilities.
Developer: RTI and the Center for Health Systems Research and Analysis, University of Wisconsin, under contract to the Centers for Medicare & Medicaid Services
Use history: Appears that indicators were identified in 2002 and were field tested.
States Currently Using: None known
Psychometric Testing: Field testing: The alpha test was designed to address the primary research questions of feasibility, reliability, and validity of the quality indicators; to include a range of provider types; and to assess feasibility and utility of the alpha test process as a model for an expanded beta test.
Summary: Indicators are intended to be applicable to both institutional settings and community-based settings. Indicators are intended to address quality at the provider level. Focus areas include dignity, health, integration and inclusion, interpersonal relationships, person-centered services and supports, respect for cultural and linguistic differences, rights, safety, self-determination, and structure.
Tool or Measure Set: Medicare Continuity Assessment Record and Evaluation (CARE)
Data Source: Assessment data collected by staff
Respondent: N/A
Mode of Data Collection: N/A
Total Items/indicators: Several hundred items, including administrative and demographic data, diagnoses, medications, treatments, and discharge needs, as well as assessment of physical and cognitive functioning.
HCBS MS Domains: Unmet Need
Populations: Medicare beneficiaries receiving post-acute care
Developer: RTI, under contract to the Centers for Medicare & Medicaid Services
Use history: Two rounds of small pilot testing have taken place. Beginning Jan. 2008 the tool will be used in a demo that involves 150 Medicare providers (acute care hospital, home health agency, nursing home, long-term care hospital and rehabilitation hospitals/units). The point of the demo is to use the uniform tool across post acute care settings and to measure labor/resource use by the different providers.
States Currently Using: None currently; CMS would like this tool to be adapted for future State use.
Psychometric Testing: Field test planned for January 2008 (see above).
Summary: The Medicare Continuity Assessment Record and Evaluation (CARE) is a uniform patient assessment instrument designed to measure differences in patient severity, resource utilization, and outcomes for patients in acute and postacute care settings. This tool will be used to (1) Standardize program information on Medicare beneficiaries' acuity at discharge from acute hospitals, (2) document medical severity, functional status and other factors related to outcomes and resource utilization at admission, discharge, and interim times during post acute treatment, and (3) understand the relationship between severity of illness, functional status, social support factors, and resource utilization. The CARE tool follows a patient across setting (skilled nursing facility, inpatient rehabilitation facility, home health agency, and long-term care hospitals) and is designed witha flexible information technology (IT) structure. One of the things CMS staff have discussed is that with a flexible IT structure and a core set of key items, the CARE tool could be a tool for States since in the future new items could be added.
Tool or Measure Set: Medicaid Home and Community-Based Waiver Program Study: Elderly/Disabled Questionnaire
Data Source: Consumer Survey
Respondent: Consumers or proxies
Mode of Data Collection: Computer-assisted telephone interviewing (CATI)
Total Items/indicators: 100+
HCBS MS Domains: Access, Quality of Life, Unmet Need, Person Centeredness/Autonomy
Populations: Elderly and disabled HCBS waiver participants
Developer: Mathematica Policy Research, under contract to the Centers for Medicare and Medicaid Services
Use history: Data collected from six States between May 2003 and June 2004 as part of federally sponsored waiver evaluation
States Currently Using: Used for evaluation purposes only.
Psychometric Testing: Article only addresses sampling and analysis, not survey development.
Summary: This tool was designed to evaluate the impact and quality of the Medicaid HCBS program. Hundreds of consumer interviews were conducted in six States. Focus areas include paid and unpaid help with ADLs and IADLs, demographics, employment, social activities, satisfaction with care and general life satisfaction.
Return to Appendix III Contents
Health Resources and Services Administration
Tool or Measure Set: State and Local Area Integrated Telephone Survey (SLAITS) National Survey of Children with Special Healthcare Needs
Data Source: Consumer Survey
Respondent: Parent or Guardian
Mode of Data Collection: Telephone Interviews
Total Items/indicators: 188 total items/questions
HCBS MS Domains: Effectiveness, Person Centeredness/Autonomy, Timeliness, Efficiency, Equity, Access, Coordination of Long-Term Care and Other Services, Unmet Need, and Quality of Life
Populations: Children with disabilities
Developer: National Center for Health Statistics of the Centers for Disease Control and Prevention. Sponsored by the Maternal and Child Health Bureau, U.S. Department of Health and Human Services
Use history: Survey conducted since 2000 through a random digit dialing process. A total of 372,174 children screened nationally and approximately 5,500 to 8,500 children in each State are identified as having special health care needs.
States Currently Using: N/A, national survey
Summary: The National Survey of Children with Special Health Care Needs is conducted by trained interviewers through a nation wide random digit dialing telephone survey process. In each State, and the District of Columbia, 750 children with special health care needs are identified to participate in the long-version of the survey. The survey is conducted in twelve different languages. Data can be reviewed at the national and State level and is used to assist in the Title V Block Grant needs assessments. Focus areas include child health and functional status; access to health care; care coordination; family-centeredness health care; and the impact of child's health status on the family.
Return to Appendix III Contents
National Institute on Disability Research and Rehabilitation
Tool or Measure Set: Assessment of Health Plans and Providers by People with Activity Limitations
Data Source: Consumer Survey
Respondent: Consumer or proxy (consumer's choice)
Mode of Data Collection: Telephone survey with mail followup; face-to-face interview if deemed necessary
Total Items/indicators: 110 items related to services and outcomes; additional demographic items and questions about proxy respondents at survey end
HCBS MS Domains: Access, Person Centeredness/Autonomy, Unmet Need, Effectiveness
Populations: Adults with activity limitations in managed care plans
Developer: Sue Palsbo, PhD, George Mason University and Peg Mastal, Delmarva Foundation Medical Care (QIO); funding provided by the National Institute of Disability Research and Rehabilitation
Use history: Tool still under development, field testing planned for Summer 2007
States Currently Using: N/A
Psychometric Testing: Ongoing
Summary: Some domains and items common to the CAHPS® tool for People with Mobility Impairment. Additional items related to breathing equipment, community integration, and services. Development included focus groups, a technical advisory group, and cognitive testing. Team members also identified 62 quality measures that can be derived from administrative databases to complement the survey, which are also undergoing field testing. Eventually, researchers will combine the data streams into composite measures, most likely to include most of the usual State Medicaid report card measures, plus some disability-sensitive measures. Focus groups of consumers reported they wanted about twice as much as in the State report cards we showed them.
Return to Appendix III Contents
Substance Abuse and Mental Health Services Administration
Tool or Measure Set: Substance Abuse and Mental Health Services Administration National Outcome Measures (NOM)
Data Source: Multiple
Respondent: N/A
Mode of Data Collection: N/A
Total Items/indicators: 26 as of June, 2007
HCBS MS Domains: Effectiveness, Person Centeredness/Autonomy, Access, Coordination of Long-Term Care and Other Services, and Quality of Life
Populations: Children with mental health disabilities and adults with severe and persistent mental illness
Developer: Center for Mental Health Services (CMHS) and collaborating partners, including States.
Use history: Released in 2004
States Currently Using: SAMHSA and the States have agreed to the goal of bringing all States to full National Outcome Measures reporting, for relevant SAMHSA grants, by the end of Fiscal Year 2007.
Psychometric Testing: Developed in collaboration with State partners to assure content validity.
Summary: The National Outcome Measures (NOM) are intended for implementation in all Substance Abuse Mental Health Services Administration (SAMHSA)-funded programs to ensure the availability of data to measure program accountability. The measures are categorized in the following focus areas: reduced morbidity, employment and education, crime and criminal justice, stability in housing, social connectedness, access and capacity, retention, perception of care, cost effectiveness, and use of evidence-based practices. Desired outcomes are provided for all focus areas. Measures intended to fall into two categories: mental health services and substance abuse (treatment and prevention) services.
Tool or Measure Set: Mental Health Statistics Improvement Program (MHSIP) Consumer Survey, Version 1. 1
Data Source: Consumer survey
Respondent: Consumer
Mode of Data Collection: Written (could be mailed or in-person)
Total Items/indicators: 28*
HCBS MS Domains: Unmet Need, Person Centeredness/Autonomy, Access, Effectiveness
Populations: Individuals receiving mental health services
Developer: Mental Health Statistics Improvement Program, supported by the Center for Mental Health Services
Use history: Surveys are available in the public domain and can be used free of charge.
States Currently Using: Community Mental Health Block Grant recipients
Psychometric Testing: Not specified
Summary: This survey instrument is one of a series of instruments developed by the Mental Health Statistics Improvement Program (MHSIP) to assist States to evaluate progress in improvement of mental health services. All items rated on a five-point scale. Items have been grouped into four primary focus areas: general satisfaction, access, quality/appropriateness, and outcomes. Survey developed in collaboration with the Center for Mental Health Services and the Substance Abuse and Mental Health Services Administration. All States are required to submit some of the data from this survey to CMHS as part of the Community Mental Health Block Grant Requirements. Several "unofficial" State versions have been posted online. There is also an MHSIP approved version for use with individuals receiving inpatient mental health services. States have modified this tool for their use and at least twelve versions exist of which factor analysis has been applied. Survey is available in Spanish.
Tool or Measure Set: Mental Health Statistics Improvement Program (MHSIP) Youth Services Survey for Families
Data Source: Consumer Survey
Respondent: Consumer or proxy (for younger children)
Mode of Data Collection: Written
Total Items/indicators: 32 (two are open-ended narrative responses)**
HCBS MS Domains: Unmet Need, Person Centeredness/Autonomy, Access, Effectiveness
Populations: Children and youth receiving mental health services
Developer: Molly Brunk, PhD, 1999, for Mental Health Statistics Improvement Program, supported by the Center for Mental Health Services
Use history: Draft version dated February 17, 2006, Survey is available in the public domain and can be used free of charge.
States Currently Using: CMHS grantees
Psychometric Testing: Not specified
Summary: This instrument was developed as part of the State Indictor Project funded by the Center for Mental Heath Services (CMHS). It was adapted from the Family Satisfaction Questionnaire used with the CMHS Comprehensive Community Mental Services for Children and their Families Program and the MHSIP Consumer Survey. Specific focus areas include choice, family involvement, access to services, quality of life, and overall satisfaction. Survey is available in Spanish.
* Version 1. 2, released February, 2006, includes 8 additional questions related to functioning and relationships. This version is called the MHSIP URS or Uniform Reporting System. The Uniform Reporting System (URS) was developed in response to the need for accountability for the expenditure of community mental health block grant funds received by States from the Federal Government. The intent of the URS tables is to allow both (1) the tracking of individual State performance over time, and (2) the aggregation of State information to develop a national picture of the public mental health systems of the States.
** The URS (Uniform Reporting System) version of the family survey includes additional items related to relationships and outcomes. The Uniform Reporting System (URS) was developed in response to the need for accountability for the expenditure of community mental health block grant funds received by States from the Federal Government. The intent of the URS tables is to allow both (1) the tracking of individual State performance over time, and (2) the aggregation of State information to develop a national picture of the public mental health systems of the States.
Return to Contents
Proceed to Next Section
