Measuring Methods

Identifying Indicators for Performance Measurement

With clear community consensus around five priorities for a stronger health care future, Reaching for Excellence set out to quantify health system performance in these areas. As a first step toward a comprehensive picture of the state of health care in Western New York, this measurement effort establishes baseline indicators for monitoring system performance and guiding quality improvement for the five priority areas. Reaching for Excellence is committed to not only tracking progress over time on current measures, but adding new and more revealing indicators of performance in priority areas as they are developed. Moreover, as the region's health system improves its performance and achieves goals, efforts may shift to improve measures in problem areas.

Measuring Performance in Priority Areas

To objectively measure and track the performance of the Western New York health system, Reaching for Excellence sought a set of data-driven indicators that met the following criteria:

  • Assesses performance in areas related to each priority
  • Representative of the patient perspective
  • Routinely measured
  • Reliable
  • Comparable across the U.S.

Fitting these criteria and serving as the foundation for this effort's indicators for Priorities #1, #2 and #3 are two national surveys assessing patient satisfaction with their health care experience in hospital settings and with care obtained through their health plan - the only national standardized source of data on such measures. The Hospital Survey and Health Plan Survey are conducted annually of hospitals and health plans across the nation as part of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) program administered by the U.S. Department of Health and Human Services. In Western New York, 24 hospitals participate in the CAHPS Hospital Survey; the three CAHPS-participating health plans evaluated in this analysis are those that serve primarily Western New York residents (Independent Health, Univera and HealthNow of New York, Inc.). Hospital patients are surveyed after they are discharged from an overnight hospital stay by mail, telephone or a mixed method (such as mail with telephone follow-up). 300 surveys a year are completed for most hospitals. Over 1,350 commercial enrollees and 1,750 Medicaid enrollees of Independent Health, Univera and HealthNow completed the Health Plan Survey in 2008. Respondents were queried over the phone, by mail and online, with the survey addressing health care services received by members in 2007.

CAHPS surveys assess quality based on reported patient experiences, which can be affected not only by the actual health care received but by patient factors such as educational level or health status. Although the CAHPS Hospital Survey results are adjusted for the patient demographic characteristics and other factors that impact patient survey responses without regard to hospital quality, CAHPS Health Plan Survey results have not been similarly adjusted. Therefore, comparison of results between plans requires assuming that enrollee demographics and other factors do not vary significantly between health plans in Western New York.

An assessment of performance in Priorities #4 and #5 are based on a mix of local and state data sources. This includes the Western New York Health Risks Assessment, conducted on behalf of the Western New York Public Health Coalition to identify health needs of the region. A total of 3,163 households comprising over 5,500 adult individuals across the eight counties of Western New York completed an HRA survey over the telephone between October 2005 and March 2005. Survey respondents were asked about their medical history, risk activities and sources of health insurance and health care.

Another significant data source was the New York State Expanded Behavioral Risk Factor Surveillance System, as reported by the New York State Department of Health County Health Assessment Indicator reports for 2004-06. A random sampling of adult respondents ages 18 and up were queried over the telephone in July 2003 about their health related behaviors. 640 interviews were conducted in each of 38 localities sampled across New York State, which included Erie, Niagara, Cattaraugus, Chautauqua, Genesee, Orleans, Allegany and Wyoning Counties.

Reaching for Excellence is committed to not only tracking progress over time on this current set of indicators, but adding new and more revealing measures of performance in priority areas as they are developed. Moreover, as the region's health system improves its performance and achieves goals, efforts may shift to improve measures in problem areas. See Reaching Excellence Together for more information on key opportunities for building a more comprehensive performance picture in priority areas.

WNY Health Care Priorities

Five key health care priorities and their regional performance (...read more)

One Friday: Four Futures

Learn how 1,700 Western New Yorkers engaged in a dialogue about the region's health care future. (...read more)