Public Report Cards — Cardiac Surgery and Beyond
http://www.100md.com
《新英格兰医药杂志》
The debate about public report cards for physicians and hospitals never seems to end.1 Some praise report cards for their role in improving the quality of care, patient safety, and the choices of patients, referring physicians, and organizations that purchase health care. Others argue that such reports can have negative consequences — for example, if physicians or hospitals, in order to protect their rankings, avoid performing surgery on severely ill patients for whom surgical treatment might otherwise be recommended. Nonetheless, more and more data are being collected and released, although policies regarding the public availability of information vary widely. Thirty-seven states and the District of Columbia have mandatory health care reporting systems for inpatient hospital data, and 10 have voluntary reporting systems, according to an August 2006 survey by the National Association of Health Data Organizations. Many states also collect emergency department and ambulatory surgery data. In general, more information is available from individual states than from any national source. An exception is data from organ transplantation programs, which are available nationally.
Cardiac surgery — most notably, coronary-artery bypass grafting (CABG) — is the longest-standing and most common focus of public report cards. Cardiac surgeons might feel that they are being subjected to a unique level of scrutiny, but the attention goes with the territory. As the Healthcare Commission in the United Kingdom, an independent inspection body, has observed, "These are big operations, which carry some risk of death. . . . Death is a very solid, objective outcome (no one can argue about it). For these reasons, heart surgery lends itself easily to analysis, even by amateurs."
CABG accounts for the majority of heart operations performed in adults in the United States; the unadjusted operative mortality rate decreased from 3.2% in 1996 to 2.2% in 2005, according to the Society of Thoracic Surgeons (see graph). Although studies have led to different conclusions,2,3 there is evidence that the public disclosure of death rates associated with surgery in New York and other states has contributed to reductions in operative mortality (which includes all deaths that occur before discharge, as well as those that occur after discharge but within 30 days after the procedure).
Unadjusted Operative Mortality Associated with Coronary-Artery Bypass Grafting in the United States, 1996–2005.
Data are from the Society of Thoracic Surgeons' adult cardiac database, spring 2006 report (prepared by the Duke Clinical Research Institute). The database currently receives data from about three quarters of all U.S. sites that perform cardiac surgery; participation rates have varied in previous years. Operative mortality includes all deaths before hospital discharge, as well as deaths after discharge but within 30 days after the procedure, unless the cause was clearly unrelated. Patients undergoing repeat surgery are included; those also undergoing aortic or mitral valve replacement are excluded.
As with other surgeries, the number of CABG procedures performed by individual surgeons is inversely related to the rate of death after CABG. This relationship accounts in large part for the association between a lower volume of cases at a given hospital and higher mortality. There are well-accepted statistical methods of adjusting for the characteristics of patients that are associated with higher mortality, such as older age, female sex, poorer heart function, the presence of other diseases, and prior open-heart surgery.
In the 1980s, some newspapers, such as the Los Angeles Times, obtained information that had been reported to state governments and published hospital-specific, risk-adjusted mortality rates for CABG and other surgeries.4 In 1990, New York became the first state to report such data publicly. Through its Cardiac Surgery Reporting System, the New York State Department of Health currently reports risk-adjusted mortality rates for CABG and heart valve surgery according to the hospital and individual surgeon, as well as similar data for coronary angioplasty. The initiative costs roughly $300,000 a year, which does not include the costs incurred by hospitals for data collection and coordination, according to Edward Hannan of the University at Albany School of Public Health, State University of New York, who serves as a consultant to the New York State Department of Health.
Other states that report on the outcomes of CABG surgery in their hospitals include Pennsylvania and New Jersey, which began such reporting in the 1990s, and Massachusetts, California, and Florida, which started more recently. New Jersey and Pennsylvania publish performance reports for individual cardiac surgeons. California is expected to begin doing so in early 2007, and Massachusetts is considering whether it, too, will publicize the patient death rates for individual surgeons.
There is no national system in the United States for disclosing the outcomes of CABG for hospitals or individual surgeons — for example, through Medicare or the Department of Veterans Affairs. The situation is different in the United Kingdom, where in April 2006, the Healthcare Commission and the Society of Cardiothoracic Surgeons of Great Britain and Ireland launched a Web site with information on the outcomes of heart surgery. The Heart Surgery in Great Britain Web site (http://heartsurgery.healthcarecommission.org.uk) provides public information about survival rates for CABG and aortic valve–replacement operations performed between April 2004 and March 2005. As of July 2006, the information covered all 37 public cardiac units in England, Scotland, and Wales, 2 new independent-sector units in England, and 173 cardiac surgeons, from 27 of the 39 units. Data for the year ending March 2006 will be available by January 2007.
In the United States, hospital-specific CABG mortality rates are usually reported annually, after a lag of 1 to 3 years. Data for individual surgeons are often reported for a period of several years. Reports may include the actual rates, or they may merely list results as "higher than expected," "same as expected," "lower than expected," or not rated because of insufficient cases. It is preferable for such reports to be based on analyses of clinical data rather than administrative data (the latter obtained from standard abstracts of patient discharge data that are compiled primarily for billing purposes), although it is easier and cheaper to analyze the administrative data. Analyses of the two kinds of data may produce different performance results for institutions and physicians: administrative data may be incomplete, may not differentiate between preexisting conditions and complications of surgery, and generally do not include important factors related to the severity of illness, such as the patient's left ventricular ejection fraction, and deaths that occur after hospital discharge.
California, Massachusetts, and New Jersey use clinical data from the Society of Thoracic Surgeons' adult cardiac database. New York uses similar data. Pennsylvania uses administrative data supplemented by clinical data, and Florida uses administrative data alone. The United Kingdom uses the logistic European System for Cardiac Operative Risk Evaluation (www.euroscore.org), which includes clinical data.
In New York, where the matter has been studied closely, there have not been major changes in the aggregate severity of illness among patients undergoing CABG, nor has there been evidence of widespread avoidance of severely ill patients, according to Hannan. A recent study showed that the New York performance reports "can reliably predict better-than-average performance by both surgeons and hospitals and can help patients and payers avoid low-performing providers."5 Patients who picked a top-performing hospital or surgeon were about half as likely to die as those who picked a hospital or surgeon with a ranking near the bottom. Surgeons whose CABG operations were associated with the highest mortality rates were much more likely than other surgeons to stop performing CABG surgery in the state within 2 years after the release of each report card (21.3% of them gave up the practice, as compared with 5.1% of other surgeons) — an effect that, among other things, highlights the need for accurate reporting. The association between a hospital's reported performance and any subsequent change in its market share was inconsistent, suggesting that the report was only one of several factors that patients, referring physicians, and health care purchasers weighed in choosing a surgeon or hospital.
Public report cards can complement nonpublic efforts to improve patient safety and the quality of care — if they are accurate, meaningful, and current. Information should be verified and selectively audited to correct mistakes and to prevent "gaming the system" by making patients appear to be at higher risk for death than they actually are. Public report cards are not going away. Indeed, they are likely to become more common and to cover both physicians and institutions, as well as additional surgeries, other procedures, and medical conditions.
Source Information
Dr. Steinbrook (rsteinbrook@attglobal.net) is a national correspondent for the Journal.
References
Werner RM, Asch DA. The unintended consequences of publicly reporting quality information. JAMA 2005;293:1239-1244.
Hannan EL, Sarrazin MS, Doran DR, Rosenthal GE. Provider profiling and quality improvement efforts in coronary artery bypass graft surgery: the effect on short-term mortality among Medicare beneficiaries. Med Care 2003;41:1164-1172.
Dranove D, Kessler D, McClellan M, Satterthwaite M. Is more information better? The effects of "report cards" on health care providers. J Polit Econ 2003;111:555-588.
Steinbrook R. Hospital quality in California. Health Aff (Millwood) 1988;7:235-236.
Jha AK, Epstein AM. The predictive accuracy of the New York State coronary artery bypass surgery report-card system. Health Aff (Millwood) 2006;25:844-855.(Robert Steinbrook, M.D.)
Cardiac surgery — most notably, coronary-artery bypass grafting (CABG) — is the longest-standing and most common focus of public report cards. Cardiac surgeons might feel that they are being subjected to a unique level of scrutiny, but the attention goes with the territory. As the Healthcare Commission in the United Kingdom, an independent inspection body, has observed, "These are big operations, which carry some risk of death. . . . Death is a very solid, objective outcome (no one can argue about it). For these reasons, heart surgery lends itself easily to analysis, even by amateurs."
CABG accounts for the majority of heart operations performed in adults in the United States; the unadjusted operative mortality rate decreased from 3.2% in 1996 to 2.2% in 2005, according to the Society of Thoracic Surgeons (see graph). Although studies have led to different conclusions,2,3 there is evidence that the public disclosure of death rates associated with surgery in New York and other states has contributed to reductions in operative mortality (which includes all deaths that occur before discharge, as well as those that occur after discharge but within 30 days after the procedure).
Unadjusted Operative Mortality Associated with Coronary-Artery Bypass Grafting in the United States, 1996–2005.
Data are from the Society of Thoracic Surgeons' adult cardiac database, spring 2006 report (prepared by the Duke Clinical Research Institute). The database currently receives data from about three quarters of all U.S. sites that perform cardiac surgery; participation rates have varied in previous years. Operative mortality includes all deaths before hospital discharge, as well as deaths after discharge but within 30 days after the procedure, unless the cause was clearly unrelated. Patients undergoing repeat surgery are included; those also undergoing aortic or mitral valve replacement are excluded.
As with other surgeries, the number of CABG procedures performed by individual surgeons is inversely related to the rate of death after CABG. This relationship accounts in large part for the association between a lower volume of cases at a given hospital and higher mortality. There are well-accepted statistical methods of adjusting for the characteristics of patients that are associated with higher mortality, such as older age, female sex, poorer heart function, the presence of other diseases, and prior open-heart surgery.
In the 1980s, some newspapers, such as the Los Angeles Times, obtained information that had been reported to state governments and published hospital-specific, risk-adjusted mortality rates for CABG and other surgeries.4 In 1990, New York became the first state to report such data publicly. Through its Cardiac Surgery Reporting System, the New York State Department of Health currently reports risk-adjusted mortality rates for CABG and heart valve surgery according to the hospital and individual surgeon, as well as similar data for coronary angioplasty. The initiative costs roughly $300,000 a year, which does not include the costs incurred by hospitals for data collection and coordination, according to Edward Hannan of the University at Albany School of Public Health, State University of New York, who serves as a consultant to the New York State Department of Health.
Other states that report on the outcomes of CABG surgery in their hospitals include Pennsylvania and New Jersey, which began such reporting in the 1990s, and Massachusetts, California, and Florida, which started more recently. New Jersey and Pennsylvania publish performance reports for individual cardiac surgeons. California is expected to begin doing so in early 2007, and Massachusetts is considering whether it, too, will publicize the patient death rates for individual surgeons.
There is no national system in the United States for disclosing the outcomes of CABG for hospitals or individual surgeons — for example, through Medicare or the Department of Veterans Affairs. The situation is different in the United Kingdom, where in April 2006, the Healthcare Commission and the Society of Cardiothoracic Surgeons of Great Britain and Ireland launched a Web site with information on the outcomes of heart surgery. The Heart Surgery in Great Britain Web site (http://heartsurgery.healthcarecommission.org.uk) provides public information about survival rates for CABG and aortic valve–replacement operations performed between April 2004 and March 2005. As of July 2006, the information covered all 37 public cardiac units in England, Scotland, and Wales, 2 new independent-sector units in England, and 173 cardiac surgeons, from 27 of the 39 units. Data for the year ending March 2006 will be available by January 2007.
In the United States, hospital-specific CABG mortality rates are usually reported annually, after a lag of 1 to 3 years. Data for individual surgeons are often reported for a period of several years. Reports may include the actual rates, or they may merely list results as "higher than expected," "same as expected," "lower than expected," or not rated because of insufficient cases. It is preferable for such reports to be based on analyses of clinical data rather than administrative data (the latter obtained from standard abstracts of patient discharge data that are compiled primarily for billing purposes), although it is easier and cheaper to analyze the administrative data. Analyses of the two kinds of data may produce different performance results for institutions and physicians: administrative data may be incomplete, may not differentiate between preexisting conditions and complications of surgery, and generally do not include important factors related to the severity of illness, such as the patient's left ventricular ejection fraction, and deaths that occur after hospital discharge.
California, Massachusetts, and New Jersey use clinical data from the Society of Thoracic Surgeons' adult cardiac database. New York uses similar data. Pennsylvania uses administrative data supplemented by clinical data, and Florida uses administrative data alone. The United Kingdom uses the logistic European System for Cardiac Operative Risk Evaluation (www.euroscore.org), which includes clinical data.
In New York, where the matter has been studied closely, there have not been major changes in the aggregate severity of illness among patients undergoing CABG, nor has there been evidence of widespread avoidance of severely ill patients, according to Hannan. A recent study showed that the New York performance reports "can reliably predict better-than-average performance by both surgeons and hospitals and can help patients and payers avoid low-performing providers."5 Patients who picked a top-performing hospital or surgeon were about half as likely to die as those who picked a hospital or surgeon with a ranking near the bottom. Surgeons whose CABG operations were associated with the highest mortality rates were much more likely than other surgeons to stop performing CABG surgery in the state within 2 years after the release of each report card (21.3% of them gave up the practice, as compared with 5.1% of other surgeons) — an effect that, among other things, highlights the need for accurate reporting. The association between a hospital's reported performance and any subsequent change in its market share was inconsistent, suggesting that the report was only one of several factors that patients, referring physicians, and health care purchasers weighed in choosing a surgeon or hospital.
Public report cards can complement nonpublic efforts to improve patient safety and the quality of care — if they are accurate, meaningful, and current. Information should be verified and selectively audited to correct mistakes and to prevent "gaming the system" by making patients appear to be at higher risk for death than they actually are. Public report cards are not going away. Indeed, they are likely to become more common and to cover both physicians and institutions, as well as additional surgeries, other procedures, and medical conditions.
Source Information
Dr. Steinbrook (rsteinbrook@attglobal.net) is a national correspondent for the Journal.
References
Werner RM, Asch DA. The unintended consequences of publicly reporting quality information. JAMA 2005;293:1239-1244.
Hannan EL, Sarrazin MS, Doran DR, Rosenthal GE. Provider profiling and quality improvement efforts in coronary artery bypass graft surgery: the effect on short-term mortality among Medicare beneficiaries. Med Care 2003;41:1164-1172.
Dranove D, Kessler D, McClellan M, Satterthwaite M. Is more information better? The effects of "report cards" on health care providers. J Polit Econ 2003;111:555-588.
Steinbrook R. Hospital quality in California. Health Aff (Millwood) 1988;7:235-236.
Jha AK, Epstein AM. The predictive accuracy of the New York State coronary artery bypass surgery report-card system. Health Aff (Millwood) 2006;25:844-855.(Robert Steinbrook, M.D.)