An investigation into general practitioners associated with high patient mortality flagged up through the Shipman inquiry: retrospective ana
http://www.100md.com
《英国医生杂志》
1 Department of Public Health and Epidemiology, University of Birmingham, Edgbaston, Birmingham B15 2TT, 2 Telford and Wrekin Primary Care Trust, Telford TF1 5RY, 3 Shropshire and Staffordshire Strategic Health Authority, Stafford ST16 3SR, 4 Shropshire County Primary Care Trust, Shrewsbury SY3 8XL
Correspondence to: M A Mohammed m.a.mohammed@bham.ac.uk
Abstract
Objective To identify a credible explanation for the excessively high mortality associated with general practitioners who were flagged up by the Shipman inquiry.
Design Retrospective analysis of routine data.
Setting Primary care.
Participants Two general practitioners in the West Midlands who were associated with an unacceptably high mortality of patients during 1993-2000.
Main outcome measures Observed and expected number of deaths and deaths in nursing homes.
Results Preliminary discussions with the general practitioners highlighted deaths in nursing homes as a possible explanatory factor. No relation was found between the expected number of deaths and deaths in nursing homes in each year during 1993-2000 for either general practitioner. In contrast, the magnitude and shape of the curves of a cumulative sum plot for excess number of deaths (observed minus expected) in each year were closely mirrored by the magnitude and shape of the curves of the number of patients dying in nursing homes; and this was reflected in the high correlations (R2 = 0.87 and 0.89) between excess mortality and the number of deaths in nursing homes in each year for the general practitioners. These findings were supported by administrative data.
Conclusions The excessively high mortality associated with two general practitioners was credibly explained by a nursing home effect. General practitioners associated with high patient mortality, albeit after sophisticated statistical analysis, should not be labelled as having poor performance but instead should be considered as a signal meriting scientific investigation.
Introduction
We adopted the pyramid of investigation model (fig 1).3 This model checks five variables—data, patient case mix, structure, process of care, and carers. We began with a review of the data. The general practitioners (A and B) were made aware of, and fully cooperated with, the investigation.
Fig 1 Pyramid model of investigation to find credible cause for high mortality of patients
Through the Shipman inquiry we obtained the raw data on mortality for the general practitioners and the analyses conducted on these data by the team from Imperial College, whose methods are fully described elsewhere.4 6 Briefly, their raw data consisted of a list of the deceased patients for each general practitioner, with personal details (date of birth, sex, postcode) and date, place, and cause of death. Their analyses included a count of observed and expected numbers of death for each year from 1993 to 2000 together with the cumulative sum plots. The expected numbers of deaths were determined using indirect standardisation adjusted for patients' age, with reference rates being derived from the relevant health authority. The cumulative sum plots were designed to detect general practitioners associated with a patient mortality of at least 4 standard deviations higher than the acceptable level. The threshold for signalling an alarm was set at 3, which was almost certain (> 99.9%) to indicate a true signal (fig 2, top). The cumulative sum plots for general practitioners A and B crossed the alarm threshold in 1996.
Fig 2 Cumulative sum plots for general practitioners A and B, with alarm threshold set at 3, (top) deaths in nursing homes compared with excess and expected deaths (middle), and XY scatter plot of deaths in nursing homes compared with excess deaths (bottom)
We used the codes for place of death in the raw data to determine the address of the place of death through the Office of National Statistics Communal Establishment File. The table shows the data obtained from the Shipman inquiry.
We undertook the generation and testing of an exploratory hypothesis using the raw and adjusted data. From these we sought specific items of data, analysis, or information that could be used to test the validity of the hypothesis.
Results
Our visual analysis of a limited dataset from the Shipman inquiry supports a "nursing home effect" for the high mortality of patients associated with certain general practitioners. We believe that our findings are credible on the basis of a graphical model, which combined the results of local knowledge, administrative data, and reported findings that patients admitted to a nursing home are known to have high mortality.7 8
Apart from the two general practitioners we investigated, we are aware of six others who also signalled "unacceptably" high patient mortality through the Shipman inquiry and whose investigations have been reported (A Rixom, personal communication, 2004).9 All six were subsequently found to have a nursing home effect. The reports describe a complex investigation process, which included several strategies encompassing quantitative and qualitative data in combination with an independent review of case notes. It remains unclear as to how useful the review of case notes was in these instances, although the methodological issues as well as time and cost are considerable.9 10 Nevertheless, our findings along with those of others, suggest a need for a replication of our analysis for the other nine general practitioners who also signalled "unacceptably" high mortality, and that the model of adjustment for case mix on which the cumulative sum plots are based may need to be refined to accommodate adequately the nursing home effect. We also perceive a need to monitor mortality in nursing homes independently.
An important implication of our finding is caution in over-interpreting data adjusted for case mix, which has been described as the case mix adjustment fallacy.11 This fallacy begins with a deceptively simple equation that relates the variance in outcomes (mortality) to a combination of three sources—chance, patient case mix, and quality of care. Sophisticated statistical techniques are often used to account for chance, both in measured and, in this case, unmeasured (by taking account of over dispersion in observed deaths) patient case mix factors, with the residual unexplained variance being prematurely assigned to quality of care.4 We have shown that such reasoning, although seductive, should be diligently avoided if the process of monitoring is to remain a credible and positive contribution to improving quality of care.12 A general practitioner associated with high patient mortality, albeit after a sophisticated analysis, should not be labelled as having poor performance but instead should be considered as meriting proper scientific investigation for a credible cause. If a national monitoring system is implemented, the need for local knowledge and expertise in interpreting the data should not be underestimated.
What is already known on this topic
After statistically sophisticated analyses, the Shipman inquiry was notified of 12 general practitioners (one being Harold Shipman) with excessively high mortality
Several of the 11 general practitioners have been investigated, mostly using the costly and challenging method of case note review
What this study adds
A "novel" pyramid model of investigation was applied to two general practitioners associated with unacceptably high patient mortality
The excess mortality is adequately explained by taking account of the proportions of patients dying in nursing homes
We thank general practitioners A and B for their cooperation and support throughout this work, the staff involved with the Shipman inquiry, Paul Aylin (Imperial College, London) for providing the relevant datasets, and A Rixom, J Billet, N Kendall, and P Old for sharing their experience.
Contributors: AR headed the investigation team. DP and HO carried out data validation checks and exploratory analysis. PM was part of the investigation team and provided guidance and support throughout and was our link with the Department of Health and the team at Imperial College, London. MAM provided a framework for the investigation, undertook analysis of the dataset from the Imperial College team, found the results shown, and wrote the first draft of the paper. AS provided guidance and support throughout. All authors contributed to the writing of the final paper. MAM will act as guarantor for the paper.
Competing interests: MAM was an expert witness at the Shipman inquiry, where he discussed the monitoring of death rates associated with general practitioners.
Ethical approval: Not required.
References
Baker R. Harold Shipman's clinical practice, 1974-1998. London: Stationery Office, 2001.
The Shipman Inquiry: independent inquiry into the issues arising from the case of Harold Fredrick Shipman. www.shipmaninquiry.org.uk/ (accessed Mar 2004).
Mohammed MA, Cheng KK, Rouse A, Marshall T. Bristol, Shipman and clinical governance: Shewhart's forgotten lessons. Lancet 2001;357: 463-7.
Aylin P, Best N, Bottle A, Marshall C. Following Shipman: a pilot system for monitoring mortality rates in primary care. Lancet 2003;362: 485-91.
Spiegelhalter D, Grigg O, Kinsman R, Treasure T. Risk-adjusted sequential probability ratio tests: applications to Bristol, Shipman and adult cardiac surgery. Int J Qual Health Care 2003;15: 7-13.
Aylin P, Best N, Bottle A, Marshall C. Monitoring of mortality rates in primary care 2003. www.the-shipmaninquiry.org.uk/documentaryday.asp?from=w&day=160 (HD 06 00001 accessed Mar 2004).
Bebbington A, Brown P, Darton R. Longitudinal study of elderly people admitted to residential and nursing homes: 30 months on. Canterbury, Kent: Personal Social Services Research Unit, University of Kent at Canterbury. www.pssru.ac.uk/pdf/P42.pdf (accessed Mar 2004).
Congdon P. Nursing home mortality: patterns and trends. Public Health Research Report No 141, Public Health Directorate, Barking and Havering Health Authority. www.bhha.org.uk/141.pdf (accessed Mar 2004).
Billett J, Kendall N, Old P. A private and confidential report. An audit of mortality in five practices in West Sussex: validation of a model for monitoring deaths in general practice. Adur, Arun and Worthing Primary Care Trust, Jun 2003.
Lilford RJ, Mohammed MA, Braunholtz D, Hofer TP. The measurement of active errors: methodological issues. Qual Safety Health Care 2003;12(suppl II): ii8-12.
Lillford RJ, Mohammed MA, Spiegelhalter D, Thomson R. Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma. Lancet 2004;363: 1147-54.
Baker R, Jones DR, Goldblatt P. Monitoring mortality rates in general practice after Shipman. BMJ 2003;326: 274-6.(Mohammed A Mohammed, seni)
Correspondence to: M A Mohammed m.a.mohammed@bham.ac.uk
Abstract
Objective To identify a credible explanation for the excessively high mortality associated with general practitioners who were flagged up by the Shipman inquiry.
Design Retrospective analysis of routine data.
Setting Primary care.
Participants Two general practitioners in the West Midlands who were associated with an unacceptably high mortality of patients during 1993-2000.
Main outcome measures Observed and expected number of deaths and deaths in nursing homes.
Results Preliminary discussions with the general practitioners highlighted deaths in nursing homes as a possible explanatory factor. No relation was found between the expected number of deaths and deaths in nursing homes in each year during 1993-2000 for either general practitioner. In contrast, the magnitude and shape of the curves of a cumulative sum plot for excess number of deaths (observed minus expected) in each year were closely mirrored by the magnitude and shape of the curves of the number of patients dying in nursing homes; and this was reflected in the high correlations (R2 = 0.87 and 0.89) between excess mortality and the number of deaths in nursing homes in each year for the general practitioners. These findings were supported by administrative data.
Conclusions The excessively high mortality associated with two general practitioners was credibly explained by a nursing home effect. General practitioners associated with high patient mortality, albeit after sophisticated statistical analysis, should not be labelled as having poor performance but instead should be considered as a signal meriting scientific investigation.
Introduction
We adopted the pyramid of investigation model (fig 1).3 This model checks five variables—data, patient case mix, structure, process of care, and carers. We began with a review of the data. The general practitioners (A and B) were made aware of, and fully cooperated with, the investigation.
Fig 1 Pyramid model of investigation to find credible cause for high mortality of patients
Through the Shipman inquiry we obtained the raw data on mortality for the general practitioners and the analyses conducted on these data by the team from Imperial College, whose methods are fully described elsewhere.4 6 Briefly, their raw data consisted of a list of the deceased patients for each general practitioner, with personal details (date of birth, sex, postcode) and date, place, and cause of death. Their analyses included a count of observed and expected numbers of death for each year from 1993 to 2000 together with the cumulative sum plots. The expected numbers of deaths were determined using indirect standardisation adjusted for patients' age, with reference rates being derived from the relevant health authority. The cumulative sum plots were designed to detect general practitioners associated with a patient mortality of at least 4 standard deviations higher than the acceptable level. The threshold for signalling an alarm was set at 3, which was almost certain (> 99.9%) to indicate a true signal (fig 2, top). The cumulative sum plots for general practitioners A and B crossed the alarm threshold in 1996.
Fig 2 Cumulative sum plots for general practitioners A and B, with alarm threshold set at 3, (top) deaths in nursing homes compared with excess and expected deaths (middle), and XY scatter plot of deaths in nursing homes compared with excess deaths (bottom)
We used the codes for place of death in the raw data to determine the address of the place of death through the Office of National Statistics Communal Establishment File. The table shows the data obtained from the Shipman inquiry.
We undertook the generation and testing of an exploratory hypothesis using the raw and adjusted data. From these we sought specific items of data, analysis, or information that could be used to test the validity of the hypothesis.
Results
Our visual analysis of a limited dataset from the Shipman inquiry supports a "nursing home effect" for the high mortality of patients associated with certain general practitioners. We believe that our findings are credible on the basis of a graphical model, which combined the results of local knowledge, administrative data, and reported findings that patients admitted to a nursing home are known to have high mortality.7 8
Apart from the two general practitioners we investigated, we are aware of six others who also signalled "unacceptably" high patient mortality through the Shipman inquiry and whose investigations have been reported (A Rixom, personal communication, 2004).9 All six were subsequently found to have a nursing home effect. The reports describe a complex investigation process, which included several strategies encompassing quantitative and qualitative data in combination with an independent review of case notes. It remains unclear as to how useful the review of case notes was in these instances, although the methodological issues as well as time and cost are considerable.9 10 Nevertheless, our findings along with those of others, suggest a need for a replication of our analysis for the other nine general practitioners who also signalled "unacceptably" high mortality, and that the model of adjustment for case mix on which the cumulative sum plots are based may need to be refined to accommodate adequately the nursing home effect. We also perceive a need to monitor mortality in nursing homes independently.
An important implication of our finding is caution in over-interpreting data adjusted for case mix, which has been described as the case mix adjustment fallacy.11 This fallacy begins with a deceptively simple equation that relates the variance in outcomes (mortality) to a combination of three sources—chance, patient case mix, and quality of care. Sophisticated statistical techniques are often used to account for chance, both in measured and, in this case, unmeasured (by taking account of over dispersion in observed deaths) patient case mix factors, with the residual unexplained variance being prematurely assigned to quality of care.4 We have shown that such reasoning, although seductive, should be diligently avoided if the process of monitoring is to remain a credible and positive contribution to improving quality of care.12 A general practitioner associated with high patient mortality, albeit after a sophisticated analysis, should not be labelled as having poor performance but instead should be considered as meriting proper scientific investigation for a credible cause. If a national monitoring system is implemented, the need for local knowledge and expertise in interpreting the data should not be underestimated.
What is already known on this topic
After statistically sophisticated analyses, the Shipman inquiry was notified of 12 general practitioners (one being Harold Shipman) with excessively high mortality
Several of the 11 general practitioners have been investigated, mostly using the costly and challenging method of case note review
What this study adds
A "novel" pyramid model of investigation was applied to two general practitioners associated with unacceptably high patient mortality
The excess mortality is adequately explained by taking account of the proportions of patients dying in nursing homes
We thank general practitioners A and B for their cooperation and support throughout this work, the staff involved with the Shipman inquiry, Paul Aylin (Imperial College, London) for providing the relevant datasets, and A Rixom, J Billet, N Kendall, and P Old for sharing their experience.
Contributors: AR headed the investigation team. DP and HO carried out data validation checks and exploratory analysis. PM was part of the investigation team and provided guidance and support throughout and was our link with the Department of Health and the team at Imperial College, London. MAM provided a framework for the investigation, undertook analysis of the dataset from the Imperial College team, found the results shown, and wrote the first draft of the paper. AS provided guidance and support throughout. All authors contributed to the writing of the final paper. MAM will act as guarantor for the paper.
Competing interests: MAM was an expert witness at the Shipman inquiry, where he discussed the monitoring of death rates associated with general practitioners.
Ethical approval: Not required.
References
Baker R. Harold Shipman's clinical practice, 1974-1998. London: Stationery Office, 2001.
The Shipman Inquiry: independent inquiry into the issues arising from the case of Harold Fredrick Shipman. www.shipmaninquiry.org.uk/ (accessed Mar 2004).
Mohammed MA, Cheng KK, Rouse A, Marshall T. Bristol, Shipman and clinical governance: Shewhart's forgotten lessons. Lancet 2001;357: 463-7.
Aylin P, Best N, Bottle A, Marshall C. Following Shipman: a pilot system for monitoring mortality rates in primary care. Lancet 2003;362: 485-91.
Spiegelhalter D, Grigg O, Kinsman R, Treasure T. Risk-adjusted sequential probability ratio tests: applications to Bristol, Shipman and adult cardiac surgery. Int J Qual Health Care 2003;15: 7-13.
Aylin P, Best N, Bottle A, Marshall C. Monitoring of mortality rates in primary care 2003. www.the-shipmaninquiry.org.uk/documentaryday.asp?from=w&day=160 (HD 06 00001 accessed Mar 2004).
Bebbington A, Brown P, Darton R. Longitudinal study of elderly people admitted to residential and nursing homes: 30 months on. Canterbury, Kent: Personal Social Services Research Unit, University of Kent at Canterbury. www.pssru.ac.uk/pdf/P42.pdf (accessed Mar 2004).
Congdon P. Nursing home mortality: patterns and trends. Public Health Research Report No 141, Public Health Directorate, Barking and Havering Health Authority. www.bhha.org.uk/141.pdf (accessed Mar 2004).
Billett J, Kendall N, Old P. A private and confidential report. An audit of mortality in five practices in West Sussex: validation of a model for monitoring deaths in general practice. Adur, Arun and Worthing Primary Care Trust, Jun 2003.
Lilford RJ, Mohammed MA, Braunholtz D, Hofer TP. The measurement of active errors: methodological issues. Qual Safety Health Care 2003;12(suppl II): ii8-12.
Lillford RJ, Mohammed MA, Spiegelhalter D, Thomson R. Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma. Lancet 2004;363: 1147-54.
Baker R, Jones DR, Goldblatt P. Monitoring mortality rates in general practice after Shipman. BMJ 2003;326: 274-6.(Mohammed A Mohammed, seni)