A Middle Ground on Public Accountability
http://www.100md.com
《新英格兰医药杂志》
The battle lines are drawn, and the disagreement between health care providers and the community of payers and purchasers is as emotional as any to date. On one side, insurance companies and employers are posting data on quality and costs for hospitals, physician groups, and even individual physicians on their Web sites. On the other, health care providers express their outrage, citing the poor quality of the administrative claims data that are used for most of these "report cards" and the failure of the analyses to adjust for socioeconomic status and important coexisting conditions. The conflict and the stakes have increased as these analyses have gone from "for your information" status to incorporation into the structure of health plans and tiered premiums and copayments.
The payer–purchaser community admits that the data are flawed but says, "Don't let perfection be the enemy of the good." Providers say that, too often, the data are downright misleading — and worse than nothing at all because patients and their families may be misdirected at a time of critical decision making. The payers and purchasers argue that the data are needed to "empower" consumers so that they can be better "shoppers" for their care. Providers see a not-so-subtle plot to justify the shifting of costs to patients and to use flawed tools of measurement to drag prestigious but expensive institutions down into apparent mediocrity.
Is there a middle ground? We believe that there is, and we propose some principles that may define it. But reaching that middle ground will require each side to understand the needs and concerns of the other.
Limitations of Claims Data
To be sure, there is fault to be found in the arguments of both sides in this debate. Some report-card "tools" are based exclusively on the analysis of medical claims — administrative data created for the purpose of getting bills paid, not for performing research into health services. Studies of the accuracy of medical claims are discouraging. For example, one review of 1003 hospitalizations for conditions coded as acute myocardial infarctions showed that 260 cases (26 percent) did not actually meet criteria for this diagnosis.1 Other research has shown that coding of chronic diseases and coexisting conditions as "secondary diagnoses" is highly variable, leading to counterintuitive findings in many analyses.2 No wonder, then, that the hospital-ratings system used in one Internet report card on health care performed poorly as a discriminator between any two individual hospitals, as compared with measures of quality based on detailed chart review.3
If claims data are often inaccurate, but randomly so, then analyses based on them are biased toward the null hypothesis — which would tend to make excellent providers and sub-par providers drift into the middle of the pack. Even more disturbing to providers is the lack of adjustment in the analyses for systematic biases, such as those due to socioeconomic status or referral patterns. For example, rankings based on mortality data that were recently published in the Boston news media suggested that the top four local hospitals for care of acute myocardial infarction were community institutions, none of which perform cardiac surgery. This analysis did not take into account the fact that these institutions routinely (and appropriately) transfer their patients at highest risk to tertiary care hospitals for immediate catheterization and revascularization. These tertiary care institutions, of course, fell farther down in the ranking. Such methodologic problems are even more severe when report cards attempt to measure the performance of individual physicians.4
Whatever their faults, these report cards do not suffer from a lack of effort or expertise. They are based on sophisticated tools designed to adjust for the severity of illness and capture a hospital's ability to rescue the sickest patients. But from the perspective of health care providers, who have high standards for the quality of information given to individual patients, the analyses too often challenge common sense. The companies behind these report cards are under tremendous pressure to deliver a list that places providers in some kind of order — and they do not let weaknesses in the data stand in the way.
Resistance from Providers
The providers' positions have problems, too. Many physicians would like to turn back the clock, arguing that the only reliable source of information for patients is their doctors. In truth, patients have never relied solely on their physicians for guidance on where to go and whom to see. And any individual physician's knowledge of where a patient can receive the best care is limited.
We live in an era in which "empowered" consumers demand transparency and will not tolerate a patronizing perspective from physicians. As patients bear an increasing proportion of their health care costs, they are likely to want to know whether evidence exists to support higher charges for the care provided by some hospitals and physicians. Advocates for consumers argue that patients should have the right to see analyses, no matter how weak in methodology, as long as the patients have been informed about the weaknesses.
Providers also have a practical reason for their reluctance to embrace public accountability for quality and efficiency, one that recalls the football adage about the dangers of passing the ball — three things can happen, and two of them are bad. Similarly, when any report card is issued, three things can happen to providers: they can appear to be better than, on a par with, or worse than their competitors. Since most patients believe that their doctors and hospitals are better than average, the risk of losing patients' confidence is, well, better than average.
Risks and Benefits of Public Reporting
Payers and purchasers often try to calm providers by asserting, "No one looks at these data anyway." In fact, there is evidence to support the belief that the report cards are too complex for most consumers and rarely influence patients' decisions about which doctors to see and where to be admitted for procedures. Data from Pennsylvania, for example, indicate that less than 1 percent of patients who were to undergo cardiac surgery knew the correct quality ratings for their physicians or hospitals.5 Such reassurances are of little comfort to providers, however, since disseminating such information is an emerging strategy for large employers, who obviously believe that patients will eventually pay attention to such information.6
Despite providers' mistrust of such data, there is also evidence that physicians and hospitals respond to public rankings — and that the resulting efforts to improve ratings can lead to improvements in health outcomes for patients.7 As physician executives in our health care delivery system, we can testify to the intense attention given to any public ranking in which our institutions appear to be only average or below average — and the high priority given to improvement if, after study, the problem appears to be real.
But there may be a darker side to the effects of public reporting. Because adjustment for the severity of illness with the use of claims data is far from a perfect science, report cards provide an incentive for physicians and hospitals to avoid the sickest patients. The unanswered question is whether providers respond to this incentive, consciously or otherwise. One recent analysis of Medicare claims data suggests that the introduction of report cards for hospitals with respect to coronary-artery bypass grafting in New York and Pennsylvania was associated with a reduction in the rates of surgery for high-risk patients, but this change was accompanied by higher net costs and worse outcomes, particularly for patients with severe conditions.8
Is public reporting dangerous for patients — because it could lead them to the wrong doctors or wrong hospitals, or because it might subtly encourage providers to "game" the system by avoiding certain patients? No one knows. There are no trials to support either hopes or fears, but both emotions have grown stronger with each recent year.
Defining a Middle Ground
When two sides feel so passionately about their positions, the wisdom of letting them use their power to fight it out is questionable. One side might carry the day, but the damage to health care and physician–patient relationships will last much longer. So the need for cultivation of a middle ground is urgent. We propose the following as core principles.
First, health care providers should give up the role of critic for that of coauthor. They should recognize that patients have a right and a need to see meaningful information about their care, and they should not argue that payers must remove these report-card tools from their Web sites. Providers should acknowledge that information itself is never inherently bad and is dangerous only if used inappropriately. Accordingly, providers should work with payers to improve the quality of these tools and to ensure that appropriate caveats about weaknesses in the analyses are on prominent display. Focusing on the reporting of measures that have gained broad acceptance, such as the core measures used by the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) and the Centers for Medicare and Medicaid Services (CMS), is a modest but important start. Payers can facilitate the transition of providers from critics to coauthors by inviting them as early as possible into the process for designing high-quality report cards.
Second, payers and employers should acknowledge the limitations of current analyses, which are based on claims data, and use a multidimensional approach to reporting on quality. Structural measures of quality such as those developed by the coalition of employers known as the Leapfrog Group are more transparent and less controversial than analyses based on claims data. This group has identified several characteristics of hospitals that are associated with better patient outcomes, such as the use of computerized order entry, the presence of intensivists in intensive care units, and a higher volume of certain high-risk procedures. As the science of quality measurement improves and additional robust clinical data become available, new measures should be incorporated. In the meantime, we recommend that payers and purchasers use at least three independent approaches to the measurement of quality in any analysis of provider performance (e.g., Leapfrog, JCAHO, and CMS measures for hospitals, and data from the Health Plan Employer Data and Information Set and board-certification and recertification status for physicians) and should explain to their members why the results may appear inconsistent.
Third, payers should use great caution as they develop tiered insurance products in which patients have to pay more for care from some providers than for care from others, either in higher copayments at the time they receive care or in higher premiums withheld from their paychecks. We do not believe that providers should be assigned to higher or lower tiers on the basis of financial factors alone. At the very least, information on quality should be provided along with the financial implications of choosing a specific hospital or doctor. We believe that a reasonable and achievable goal is to develop approaches that assign providers to tiers on the basis of data on both efficiency and quality. For physicians, such measures might include the percentage of prescriptions they write that are for generic medications and their performance on HEDIS measures. For hospitals, these measures might include the average length of stay and Leapfrog criteria such as the use of computerized order-entry systems or the use of intensivists in intensive care units. We do not believe that currently available claims-based report cards are reliable enough to be used to place providers in different tiers with regard to insurance coverage. The contributions of individual measures of quality and efficiency in the assignment of providers to tiers should be transparent to purchasers and patients so that they can use their own values to make their choices.
Fourth, the quality and efficiency of care provided by physicians should be analyzed at the group level (e.g., 10 or more physicians) with adequate adjustment for patients' health status and other factors that are likely to influence performance. We believe that considerable danger arises from the publication of data on individual physicians or from the use of such data to include or exclude their services from insurance coverage. We think that data on individual physicians should be provided to those physicians so that they can try to improve their quality and efficiency but that these data should not be used for other purposes because the samples are too small and adjustment for severity of illness and socioeconomic status of treated patients is inadequate. Publication or other use of data on individual physicians introduces too great an inducement to physicians to avoid providing care for low-income patients or those with severe or costly conditions.
Finally, we would argue for collaboration among payers, purchasers, patients, and providers in the development of systems of public accountability. There are a large number of potential report cards available to patients with specific illnesses, and they often give inconsistent results — causing confusion among patients and despair among providers. Each payer in a region does not really need to develop its own version of a quality report card on providers. Collectively, we should try to converge on a smaller number of more meaningful measures to reduce the burden and costs of data collection. As we do so, we owe our patients clear and complete explanations of the strengths and weaknesses of the information that they are receiving. None of these measures are perfect, and we should give both doctors and patients more experience with these data before they are used in insurance products. This experience will help patients make choices among insurance options and help physicians understand how to improve the care they provide.
We think these basic principles characterize a reasonable middle ground that accommodates the needs and fears of payers, purchasers, and health care providers. The ability of both sides to occupy this middle ground is likely to influence their ability to collaborate on solving other daunting issues facing health care today. After all, real progress in addressing rising costs, gaps in the quality of care, and problems with patient safety only begins with the measurement and description of the issues. Payers and providers must invest both resources and time in the systems that can improve health care — and this combined investment will occur more naturally if payers and providers do not believe they are necessarily adversaries.
Source Information
From Partners Healthcare System (T.H.L.), Massachusetts General Hospital (G.S.M.), Brigham and Women's Hospital (T.A.B.), Harvard Medical School (T.H.L., G.S.M., T.A.B.), and the Harvard School of Public Health (T.A.B.) — all in Boston.
Address reprint requests to Dr. Lee at Partners Healthcare System, Prudential Tower, 11th Fl., 800 Boylston St., Boston, MA 02199, or at thlee@partners.org.
References
Iezzoni LI, Burnside S, Sickles L, Moskowitz MA, Sawitz E, Levine PA. Coding of acute myocardial infarction: clinical and policy implications. Ann Intern Med 1988;109:745-751.
Iezzoni LI, Foley SM, Daley J, Hughes J, Fisher ES, Heeren T. Comorbidities, complications, and coding bias: does the number of diagnosis codes matter in predicting in-hospital mortality? JAMA 1992;267:2197-2203.
Krumholz HM, Rathore SS, Chen J, Wang Y, Radford MJ. Evaluation of a consumer-oriented Internet health care report card: the risk of quality ratings based on mortality data. JAMA 2002;287:1277-1287.
Landon BE, Normand S-LT, Blumenthal D, Daley J. Physician clinical performance assessment: prospects and barriers. JAMA 2003;290:1183-1189.
Schneider EC, Epstein AM. Use of public performance reports: a survey of patients undergoing cardiac surgery. JAMA 1998;279:1638-1642.
Galvin R, Milstein A. Large employers' new strategies in health care. N Engl J Med 2002;347:939-942.
Marshall MN, Shekelle PG, Leatherman S, Brook RH. The public release of performance data: what do we expect to gain? A review of the evidence. JAMA 2000;283:1866-1874.
Dranove D, Kessler D, McClellan M, Satterthwaite M. Is more information better? The effects of "report cards" on health care providers. J Polit Econ 2003;111:555-588.
Related Letters:
A Middle Ground on Public Accountability
Galvin R. S., Delbanco S., de Brantes F., Volavka M. P., Gorton C. P., Lee T. H., Meyer G. S., Brennan T. A.(Thomas H. Lee, M.D., Greg)
The payer–purchaser community admits that the data are flawed but says, "Don't let perfection be the enemy of the good." Providers say that, too often, the data are downright misleading — and worse than nothing at all because patients and their families may be misdirected at a time of critical decision making. The payers and purchasers argue that the data are needed to "empower" consumers so that they can be better "shoppers" for their care. Providers see a not-so-subtle plot to justify the shifting of costs to patients and to use flawed tools of measurement to drag prestigious but expensive institutions down into apparent mediocrity.
Is there a middle ground? We believe that there is, and we propose some principles that may define it. But reaching that middle ground will require each side to understand the needs and concerns of the other.
Limitations of Claims Data
To be sure, there is fault to be found in the arguments of both sides in this debate. Some report-card "tools" are based exclusively on the analysis of medical claims — administrative data created for the purpose of getting bills paid, not for performing research into health services. Studies of the accuracy of medical claims are discouraging. For example, one review of 1003 hospitalizations for conditions coded as acute myocardial infarctions showed that 260 cases (26 percent) did not actually meet criteria for this diagnosis.1 Other research has shown that coding of chronic diseases and coexisting conditions as "secondary diagnoses" is highly variable, leading to counterintuitive findings in many analyses.2 No wonder, then, that the hospital-ratings system used in one Internet report card on health care performed poorly as a discriminator between any two individual hospitals, as compared with measures of quality based on detailed chart review.3
If claims data are often inaccurate, but randomly so, then analyses based on them are biased toward the null hypothesis — which would tend to make excellent providers and sub-par providers drift into the middle of the pack. Even more disturbing to providers is the lack of adjustment in the analyses for systematic biases, such as those due to socioeconomic status or referral patterns. For example, rankings based on mortality data that were recently published in the Boston news media suggested that the top four local hospitals for care of acute myocardial infarction were community institutions, none of which perform cardiac surgery. This analysis did not take into account the fact that these institutions routinely (and appropriately) transfer their patients at highest risk to tertiary care hospitals for immediate catheterization and revascularization. These tertiary care institutions, of course, fell farther down in the ranking. Such methodologic problems are even more severe when report cards attempt to measure the performance of individual physicians.4
Whatever their faults, these report cards do not suffer from a lack of effort or expertise. They are based on sophisticated tools designed to adjust for the severity of illness and capture a hospital's ability to rescue the sickest patients. But from the perspective of health care providers, who have high standards for the quality of information given to individual patients, the analyses too often challenge common sense. The companies behind these report cards are under tremendous pressure to deliver a list that places providers in some kind of order — and they do not let weaknesses in the data stand in the way.
Resistance from Providers
The providers' positions have problems, too. Many physicians would like to turn back the clock, arguing that the only reliable source of information for patients is their doctors. In truth, patients have never relied solely on their physicians for guidance on where to go and whom to see. And any individual physician's knowledge of where a patient can receive the best care is limited.
We live in an era in which "empowered" consumers demand transparency and will not tolerate a patronizing perspective from physicians. As patients bear an increasing proportion of their health care costs, they are likely to want to know whether evidence exists to support higher charges for the care provided by some hospitals and physicians. Advocates for consumers argue that patients should have the right to see analyses, no matter how weak in methodology, as long as the patients have been informed about the weaknesses.
Providers also have a practical reason for their reluctance to embrace public accountability for quality and efficiency, one that recalls the football adage about the dangers of passing the ball — three things can happen, and two of them are bad. Similarly, when any report card is issued, three things can happen to providers: they can appear to be better than, on a par with, or worse than their competitors. Since most patients believe that their doctors and hospitals are better than average, the risk of losing patients' confidence is, well, better than average.
Risks and Benefits of Public Reporting
Payers and purchasers often try to calm providers by asserting, "No one looks at these data anyway." In fact, there is evidence to support the belief that the report cards are too complex for most consumers and rarely influence patients' decisions about which doctors to see and where to be admitted for procedures. Data from Pennsylvania, for example, indicate that less than 1 percent of patients who were to undergo cardiac surgery knew the correct quality ratings for their physicians or hospitals.5 Such reassurances are of little comfort to providers, however, since disseminating such information is an emerging strategy for large employers, who obviously believe that patients will eventually pay attention to such information.6
Despite providers' mistrust of such data, there is also evidence that physicians and hospitals respond to public rankings — and that the resulting efforts to improve ratings can lead to improvements in health outcomes for patients.7 As physician executives in our health care delivery system, we can testify to the intense attention given to any public ranking in which our institutions appear to be only average or below average — and the high priority given to improvement if, after study, the problem appears to be real.
But there may be a darker side to the effects of public reporting. Because adjustment for the severity of illness with the use of claims data is far from a perfect science, report cards provide an incentive for physicians and hospitals to avoid the sickest patients. The unanswered question is whether providers respond to this incentive, consciously or otherwise. One recent analysis of Medicare claims data suggests that the introduction of report cards for hospitals with respect to coronary-artery bypass grafting in New York and Pennsylvania was associated with a reduction in the rates of surgery for high-risk patients, but this change was accompanied by higher net costs and worse outcomes, particularly for patients with severe conditions.8
Is public reporting dangerous for patients — because it could lead them to the wrong doctors or wrong hospitals, or because it might subtly encourage providers to "game" the system by avoiding certain patients? No one knows. There are no trials to support either hopes or fears, but both emotions have grown stronger with each recent year.
Defining a Middle Ground
When two sides feel so passionately about their positions, the wisdom of letting them use their power to fight it out is questionable. One side might carry the day, but the damage to health care and physician–patient relationships will last much longer. So the need for cultivation of a middle ground is urgent. We propose the following as core principles.
First, health care providers should give up the role of critic for that of coauthor. They should recognize that patients have a right and a need to see meaningful information about their care, and they should not argue that payers must remove these report-card tools from their Web sites. Providers should acknowledge that information itself is never inherently bad and is dangerous only if used inappropriately. Accordingly, providers should work with payers to improve the quality of these tools and to ensure that appropriate caveats about weaknesses in the analyses are on prominent display. Focusing on the reporting of measures that have gained broad acceptance, such as the core measures used by the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) and the Centers for Medicare and Medicaid Services (CMS), is a modest but important start. Payers can facilitate the transition of providers from critics to coauthors by inviting them as early as possible into the process for designing high-quality report cards.
Second, payers and employers should acknowledge the limitations of current analyses, which are based on claims data, and use a multidimensional approach to reporting on quality. Structural measures of quality such as those developed by the coalition of employers known as the Leapfrog Group are more transparent and less controversial than analyses based on claims data. This group has identified several characteristics of hospitals that are associated with better patient outcomes, such as the use of computerized order entry, the presence of intensivists in intensive care units, and a higher volume of certain high-risk procedures. As the science of quality measurement improves and additional robust clinical data become available, new measures should be incorporated. In the meantime, we recommend that payers and purchasers use at least three independent approaches to the measurement of quality in any analysis of provider performance (e.g., Leapfrog, JCAHO, and CMS measures for hospitals, and data from the Health Plan Employer Data and Information Set and board-certification and recertification status for physicians) and should explain to their members why the results may appear inconsistent.
Third, payers should use great caution as they develop tiered insurance products in which patients have to pay more for care from some providers than for care from others, either in higher copayments at the time they receive care or in higher premiums withheld from their paychecks. We do not believe that providers should be assigned to higher or lower tiers on the basis of financial factors alone. At the very least, information on quality should be provided along with the financial implications of choosing a specific hospital or doctor. We believe that a reasonable and achievable goal is to develop approaches that assign providers to tiers on the basis of data on both efficiency and quality. For physicians, such measures might include the percentage of prescriptions they write that are for generic medications and their performance on HEDIS measures. For hospitals, these measures might include the average length of stay and Leapfrog criteria such as the use of computerized order-entry systems or the use of intensivists in intensive care units. We do not believe that currently available claims-based report cards are reliable enough to be used to place providers in different tiers with regard to insurance coverage. The contributions of individual measures of quality and efficiency in the assignment of providers to tiers should be transparent to purchasers and patients so that they can use their own values to make their choices.
Fourth, the quality and efficiency of care provided by physicians should be analyzed at the group level (e.g., 10 or more physicians) with adequate adjustment for patients' health status and other factors that are likely to influence performance. We believe that considerable danger arises from the publication of data on individual physicians or from the use of such data to include or exclude their services from insurance coverage. We think that data on individual physicians should be provided to those physicians so that they can try to improve their quality and efficiency but that these data should not be used for other purposes because the samples are too small and adjustment for severity of illness and socioeconomic status of treated patients is inadequate. Publication or other use of data on individual physicians introduces too great an inducement to physicians to avoid providing care for low-income patients or those with severe or costly conditions.
Finally, we would argue for collaboration among payers, purchasers, patients, and providers in the development of systems of public accountability. There are a large number of potential report cards available to patients with specific illnesses, and they often give inconsistent results — causing confusion among patients and despair among providers. Each payer in a region does not really need to develop its own version of a quality report card on providers. Collectively, we should try to converge on a smaller number of more meaningful measures to reduce the burden and costs of data collection. As we do so, we owe our patients clear and complete explanations of the strengths and weaknesses of the information that they are receiving. None of these measures are perfect, and we should give both doctors and patients more experience with these data before they are used in insurance products. This experience will help patients make choices among insurance options and help physicians understand how to improve the care they provide.
We think these basic principles characterize a reasonable middle ground that accommodates the needs and fears of payers, purchasers, and health care providers. The ability of both sides to occupy this middle ground is likely to influence their ability to collaborate on solving other daunting issues facing health care today. After all, real progress in addressing rising costs, gaps in the quality of care, and problems with patient safety only begins with the measurement and description of the issues. Payers and providers must invest both resources and time in the systems that can improve health care — and this combined investment will occur more naturally if payers and providers do not believe they are necessarily adversaries.
Source Information
From Partners Healthcare System (T.H.L.), Massachusetts General Hospital (G.S.M.), Brigham and Women's Hospital (T.A.B.), Harvard Medical School (T.H.L., G.S.M., T.A.B.), and the Harvard School of Public Health (T.A.B.) — all in Boston.
Address reprint requests to Dr. Lee at Partners Healthcare System, Prudential Tower, 11th Fl., 800 Boylston St., Boston, MA 02199, or at thlee@partners.org.
References
Iezzoni LI, Burnside S, Sickles L, Moskowitz MA, Sawitz E, Levine PA. Coding of acute myocardial infarction: clinical and policy implications. Ann Intern Med 1988;109:745-751.
Iezzoni LI, Foley SM, Daley J, Hughes J, Fisher ES, Heeren T. Comorbidities, complications, and coding bias: does the number of diagnosis codes matter in predicting in-hospital mortality? JAMA 1992;267:2197-2203.
Krumholz HM, Rathore SS, Chen J, Wang Y, Radford MJ. Evaluation of a consumer-oriented Internet health care report card: the risk of quality ratings based on mortality data. JAMA 2002;287:1277-1287.
Landon BE, Normand S-LT, Blumenthal D, Daley J. Physician clinical performance assessment: prospects and barriers. JAMA 2003;290:1183-1189.
Schneider EC, Epstein AM. Use of public performance reports: a survey of patients undergoing cardiac surgery. JAMA 1998;279:1638-1642.
Galvin R, Milstein A. Large employers' new strategies in health care. N Engl J Med 2002;347:939-942.
Marshall MN, Shekelle PG, Leatherman S, Brook RH. The public release of performance data: what do we expect to gain? A review of the evidence. JAMA 2000;283:1866-1874.
Dranove D, Kessler D, McClellan M, Satterthwaite M. Is more information better? The effects of "report cards" on health care providers. J Polit Econ 2003;111:555-588.
Related Letters:
A Middle Ground on Public Accountability
Galvin R. S., Delbanco S., de Brantes F., Volavka M. P., Gorton C. P., Lee T. H., Meyer G. S., Brennan T. A.(Thomas H. Lee, M.D., Greg)