Development and Testing of the CAHPS Hospital Survey 

Back to top

The Initial Input for Measures

In July of 2002, the Agency for Healthcare Research and Quality (AHRQ) published a "Call for Measures" in the Federal Register that asked organizations to suggest survey items for the CAHPS development team to consider. Seven organizations responded by providing instruments and items for consideration.

The CAHPS team also drew on other sources, including:

• Items from the CAHPS Health Plan Survey for adults;
• Web chat questions and comments (October 24, 2002);
• Stakeholders' meeting (November 7, 2002);
• Vendors' meeting (November 18, 2002);
• Responses to an electronic mailbox for the CAHPS Hospital Survey;
• Literature review conducted by the RAND CAHPS Team; and
• Results of cognitive testing of the instrument (see below).

More information on sources for the Hospital Survey.

Back to top

OMB Clearance of the Initial Draft

Using the input received from these sources, the CAHPS Team developed a draft of the Hospital Survey instrument and submitted it to Centers for Medicare & Medicaid Services (CMS) on January 15, 2003. CMS then submitted this draft to the U.S. Office of Management and Budget (OMB) for clearance.

As part of the OMB clearance process, CMS published a Federal Register notice on June 27, 2003, asking for comments on the draft instrument as well as input about implementation options. CMS and AHRQ responded to these comments in accordance with OMB's Paperwork Reduction Act requirements. Since that time, four other notices have been published in the Federal Register, providing ample opportunity for public input. The fifth and final notice was published in the summer of 2005.

Back to top

The Testing Process

The instrument has been through extensive testing since the fall of 2003.

Pilot testing. CMS conducted pilot tests of the draft instrument through a contract with the Quality Improvement Organizations (QIOs) in three states: Arizona, Maryland, and New York. This pilot test included 132 hospitals and resulted in over 19,000 completed surveys. Testing began in June 2003 and ended in August 2003. The report from this study is available at http://www.cms.hhs.gov/HospitalQualityInits/downloads/
Hospital3State_Pilot_Analysis_Final200512.pdf.

Focus groups. AHRQ and CMS conducted six focus groups with consumers in October 2003 and another 10 in March 2004. These focus groups were conducted in four cities and included adults who had recently been in a hospital or who were caregivers for someone who had recently been in a hospital.

Additional field testing. Over a 6-month period beginning in the fall of 2003, the CAHPS Consortium tested the instrument in five volunteer sites encompassing over 375 hospitals:

• Calgary Health Region;
• California Institute for Health System Performance;
• California Regions of Kaiser Permanente;
• Massachusetts General Hospital; and
• Premier Incorporated.

The CAHPS Team used these field tests to learn more about the instrument, sampling processes, data collection processes, and other issues related to survey implementation. For more information about these five test sites, visit Voluntary Testing of Hospital CAHPS.

Pre-implementation testing. In the summer of 2004, the CAHPS Consortium worked with hospitals and vendors that volunteered to test the current instrument in order to identify ways to minimize the potential burden and disruption posed by this survey. Working with these test sites, researchers investigated various approaches to integrating the survey items into existing questionnaires as well as alternative protocols for administering the survey.

More pre-implementation testing took place in the spring and summer of 2005. Participants used this opportunity to integrate their own items into the CAHPS Hospital Survey and experiment with other issues related to sampling and survey administration.

Back to top

NQF's Endorsement of the Recommended Instrument

In the fall of 2004, having concluded the instrument testing process, AHRQ and the CAHPS Consortium gave CMS their recommendations for the questionnaire and administration guidelines. CMS submitted a 25-item instrument to the National Quality Forum's (NQF) review and consensus-building process for endorsement. An NQF committee has made some recommendations, including a request to add back two items, to which CMS has responded. After a period of public review and comment on the 27-item version, the NQF membership endorsed the CAHPS Hospital Survey as a measure in May 2005.

Back to top

Archived Documents

• Measuring Patient's Hospital Care Experiences
• H-CAHPS Stakeholder Committee Meeting, November 20, 2003
• H-CAHPS Stakeholder Committee Meeting, November 7, 2002
• H-CAHPS: Web Chat Transcript
• Meeting Summary, May 9, 2002

Back to top