Not everything that counts can be counted; not everything that can be counted counts
http://www.100md.com
《英国医生杂志》
1 London School of Hygiene and Tropical Medicine, London WC1E 7HT martin.mckee@lshtm.ac.uk
Few examples better show the triumph of ideology over evidence than the continuing quest to encourage patient choice by publishing the outcomes of healthcare providers.1 Perhaps because the stated objectives seem so self evidently reasonable—providing information to the public who pay for and use health services and supporting patients' ability to choose where they will be treated—opposition to this idea from sceptics is difficult without being accused of paternalism or worse. But the task of improving health care by publishing outcomes is far from simple.2-4 Essentially there are at least three problems. The first is to develop a means of assessing outcomes that provides comparable information which allows patients reliably to differentiate good and bad performers, adequately capturing differences in case mix and with sufficient power that differences do not arise by chance.5 The second is to embed this information within a system that leads to genuine improvements in quality by those underperforming, rather than opportunistic behaviour in relation to either recording6 or work undertaken,7 designed solely to improve what is reported which often makes things worse. The culture of often meaningless targets within the NHS is throwing up new examples of the latter almost every week. The most recent is the way in which hospital emergency departments, anxious not to exceed the target for patients to wait no longer than four hours before being admitted or discharged, are now refusing to admit patients from waiting ambulances until they are ready to be seen. Ambulance trusts, whose vehicles are tied up in queues outside hospitals, are investing in inflatable tents into which their patients can be deposited, in a kind of target-free limbo.8
It is the third set of problems related to what the information actually tell us about healthcare providers which Broder and colleagues investigate in their paper from California.1 Even in a setting where the amount of investment in information technology can only be dreamed of by those working in most other countries, the published data cover at most only 12% of procedures. And by looking only at procedures the data ignore the vast amount of care that does not involve one. In other words, such systems capture only a tiny amount of the overall work of a healthcare provider. The information is also largely out of date. Given the rapid pace of change in health care, how useful is it to know, when seeking treatment now, how a provider was doing five years ago?
Although the British public is already ambivalent about the value of such information,9 this paper is unlikely to deter those policy makers whose faith in the benefits of publishing the outcomes of healthcare providers is unshakeable by reason, although it may help to inform those who are undecided. In coming to a view they might refer to a sign that Einstein kept on his wall: "Not everything that counts can be counted; not everything that can be counted counts."
Competing interests: None declared.
References
Broder MS, Payne-Simon L, Brook RH. Measures of surgical quality: what will patients know by 2005? BMJ 2004;328: 152-3.
Jacobson B, Mindell J, McKee M. Hospital mortality league tables. BMJ 2003;326: 777-8.
Leyland AH, Boddy FA. League tables and acute myocardial infarction. Lancet 1998;351: 555-8.
Goldstein H, Spiegelhalter DJ. League tables and their limitations: statistical issues in comparisons of institutional performance. J R Stat Soc A 1996;159: 385-443.
Marshall EC, Spiegelhalter DJ. Reliability of league tables of in vitro fertilisation clinics: retrospective analysis of live birth rates. BMJ 1998;316: 1701-4.
Green J, Wintfeld N. Report cards on cardiac surgeons: assessing New York State's approach. N Engl J Med 1995;332: 1229-32.
Schneider EC, Epstein AM. Influence of cardiac-surgery performance reports on referral practices and access to care: a survey of cardiovascular specialists. N Engl J Med 1996;335: 251-6.
BBC. Tents prepared for A&E patients. London: BBC, 2003. http://news.bbc.co.uk/1/hi/health/3235349.stm (accessed 30 Dec 2003).
Magee H, Davis L-J, Coulter A. Public views on healthcare performance indicators and patient choice. J R Soc Med 2003;96: 338-42.(Martin McKee, professor o)
Few examples better show the triumph of ideology over evidence than the continuing quest to encourage patient choice by publishing the outcomes of healthcare providers.1 Perhaps because the stated objectives seem so self evidently reasonable—providing information to the public who pay for and use health services and supporting patients' ability to choose where they will be treated—opposition to this idea from sceptics is difficult without being accused of paternalism or worse. But the task of improving health care by publishing outcomes is far from simple.2-4 Essentially there are at least three problems. The first is to develop a means of assessing outcomes that provides comparable information which allows patients reliably to differentiate good and bad performers, adequately capturing differences in case mix and with sufficient power that differences do not arise by chance.5 The second is to embed this information within a system that leads to genuine improvements in quality by those underperforming, rather than opportunistic behaviour in relation to either recording6 or work undertaken,7 designed solely to improve what is reported which often makes things worse. The culture of often meaningless targets within the NHS is throwing up new examples of the latter almost every week. The most recent is the way in which hospital emergency departments, anxious not to exceed the target for patients to wait no longer than four hours before being admitted or discharged, are now refusing to admit patients from waiting ambulances until they are ready to be seen. Ambulance trusts, whose vehicles are tied up in queues outside hospitals, are investing in inflatable tents into which their patients can be deposited, in a kind of target-free limbo.8
It is the third set of problems related to what the information actually tell us about healthcare providers which Broder and colleagues investigate in their paper from California.1 Even in a setting where the amount of investment in information technology can only be dreamed of by those working in most other countries, the published data cover at most only 12% of procedures. And by looking only at procedures the data ignore the vast amount of care that does not involve one. In other words, such systems capture only a tiny amount of the overall work of a healthcare provider. The information is also largely out of date. Given the rapid pace of change in health care, how useful is it to know, when seeking treatment now, how a provider was doing five years ago?
Although the British public is already ambivalent about the value of such information,9 this paper is unlikely to deter those policy makers whose faith in the benefits of publishing the outcomes of healthcare providers is unshakeable by reason, although it may help to inform those who are undecided. In coming to a view they might refer to a sign that Einstein kept on his wall: "Not everything that counts can be counted; not everything that can be counted counts."
Competing interests: None declared.
References
Broder MS, Payne-Simon L, Brook RH. Measures of surgical quality: what will patients know by 2005? BMJ 2004;328: 152-3.
Jacobson B, Mindell J, McKee M. Hospital mortality league tables. BMJ 2003;326: 777-8.
Leyland AH, Boddy FA. League tables and acute myocardial infarction. Lancet 1998;351: 555-8.
Goldstein H, Spiegelhalter DJ. League tables and their limitations: statistical issues in comparisons of institutional performance. J R Stat Soc A 1996;159: 385-443.
Marshall EC, Spiegelhalter DJ. Reliability of league tables of in vitro fertilisation clinics: retrospective analysis of live birth rates. BMJ 1998;316: 1701-4.
Green J, Wintfeld N. Report cards on cardiac surgeons: assessing New York State's approach. N Engl J Med 1995;332: 1229-32.
Schneider EC, Epstein AM. Influence of cardiac-surgery performance reports on referral practices and access to care: a survey of cardiovascular specialists. N Engl J Med 1996;335: 251-6.
BBC. Tents prepared for A&E patients. London: BBC, 2003. http://news.bbc.co.uk/1/hi/health/3235349.stm (accessed 30 Dec 2003).
Magee H, Davis L-J, Coulter A. Public views on healthcare performance indicators and patient choice. J R Soc Med 2003;96: 338-42.(Martin McKee, professor o)