Clustering by health professional in individually randomised trials
http://www.100md.com
《英国医生杂志》
1 MRC Biostatistics Unit, Institute of Public Health, Cambridge CB2 2SR
Correspondence to: K J Lee kjl27@cam.ac.uk
Patient outcomes in many randomised trials depend crucially on the health professional delivering the intervention, but the resulting clustering is rarely considered in the analysis
Introduction
We reviewed all trials randomising individuals published in the BMJ during 2002, thus covering a wide range of medical areas. Trials randomising groups of individuals (cluster randomised trials) raise different issues and so were excluded.3 We recorded information on:
The type of clustering present—imposed or natural
Whether the issue was recognised (irrespective of whether it was accounted for in the statistical analysis)
How important we thought the clustering was, and
Whether we felt it had been adequately accounted for in the statistical analysis.
The final two items are subjective. Clustering judged as likely to be important was directly related to the outcomes in the trial (such as a therapist delivering an intervention). All multicentre trials were recorded as having natural clustering irrespective of the number of centres. In a validation exercise, a second reader independently reviewed a random sample of seven trials; there was 95% agreement on the items recorded.
We identified 42 trials (see bmj.com for list). Thirty eight had some form of clustering, with 17 (40%) having clustering by health professional imposed by the design of the trial (table 1). Only six out of the 38 trials mentioned clustering as a potential issue. Four of these allowed in some way for clustering in the analysis of the trial's results, although three of the four failed to recognise multiple sources of clustering.
Table 1 Assessment of clustering in 42 individually randomised trials published in BMJ in 2002
We classified clustering as unlikely to affect the results in 20 trials; four had no clustering (either single centre trials or trials where the intervention was delivered by a single health professional); and the remaining 16 were trials with natural clustering not directly related to the outcomes being assessed. Nineteen trials (45%) showed clustering that was likely to affect their results. Of these, only one attempted to take the issue into account. This trial, of community nurses specialising in Parkinson's disease,4 explicitly investigated the potential variability in results between nurses, although the researchers found it insignificant and disregarded it in the final analysis. We conclude that some potential for clustering exists in almost all trials, with over a third of trials having clustering by health professional imposed by the design. This clustering is infrequently acknowledged, and even more rarely adequately addressed.
Is clustering important?
Our analysis shows that potential clustering of outcomes is common in trials that randomise individuals but is usually ignored in the analysis of results. A review of trials in psychotherapy research also found that two thirds of trials ignored clustering.6 Clustering is most likely to have an important effect on results when health professionals actively deliver the intervention. Patients' outcomes may then depend crucially on the skill and enthusiasm of the professional involved. This may also be an issue in cluster randomised trials in which, although the clustering used in the randomisation process is recognised and generally adjusted for, further important forms of clustering may not be identified.
In the teleconsultation trial, the intervention effect became non-significant once clustering had been accounted for. Similar results have been reported in a trial comparing the delivery of minor acute care by the patient's general practitioner with that of commercial deputising services,7 in allowing for physician level clustering on the quality of diabetes care between specialty groups,8 and in studies of small group teaching.9
Summary points
Clustering is common in individually randomised trials
The potential effects of clustering are generally ignored in the analysis of trial results
Clustering is particularly important when interventions are delivered by more than one health professional
Ignoring clustering can lead to incorrect conclusions
Bigger sample sizes are needed to accommodate the potential for clustering
Clustering reduces the effective sample size, reducing the power of a trial to detect an intervention effect.7 10 Thus the best way to deal with the problem is to anticipate it at the time of design and to increase the sample size.9 We do not recommend trying to identify clustering on the basis of a statistical test of significance. Such tests lack power, and a nonsignificant result does not rule out the presence of important clustering.11 Rather, clustering should be anticipated on the basis of a trial's design, and so making allowance for it should follow as a consequence. Similar arguments apply both to cluster randomised trials3 and to individually randomised multicentre or international trials.12-14
Clustering also affects the generalisability of conclusions. For example, in therapy trials, the sample of therapists in the trial should be representative of those who are going to deliver the intervention in practice. Even if this is the case, the analysis must acknowledge the clusters for the conclusions to be justified.6 The issue of clustering of outcomes in randomised trials warrants much more attention than it has received so far, not only in design and analysis, but also in drawing justified and generalisable conclusions.
References for trials and details of clustering and the statistical analysis are on bmj.com
We thank the virtual outreach project team, in particular Paul Wallace and Julie Barber, for allowing us access to the data from the teleconsultation trial.
Contributors and sources: KJL is undertaking research on the topic of clustering in individually randomised trials. SGT has extensive experience in applied and methodological clinical trial research. SGT proposed the investigation. KJL reviewed the BMJ papers and reanalysed the teleconsultation trial. Both authors participated in drafting the article. KJL is guarantor.
Competing interests: None declared.
References
Dey P, Bundred N, Gibbs A, Hopwood P, Baildam A, Boggis C, et al. Costs and benefits of a one stop clinic compared with a dedicated breast clinic: randomised controlled trial. BMJ 2002;324: 507-11.
Koning S, Suijlekom-Smit LW, Nouwen JL, Verduin CM, Bernsen RM, Oranje AP, et al. Fusidic acid cream in the treatment of impetigo in general practice: double blind randomised placebo controlled trial. BMJ 2002;324: 203-6.
Donner A, Klar N. Design and analysis of cluster randomization trials in health research. London: Arnold, 2000.
Jarman B, Hurwitz B, Cook A, Bajekal M, Lee A. Effects of community based nurses specialising in Parkinson's disease on health outcome and costs: randomised controlled trial. BMJ 2002;324: 1072-5.
Wallace P, Haines A, Harrison R, Barber J, Thompson SG, Jacklin P, et al. Joint teleconsultations (virtual outreach) versus standard outpatient appointments for patients referred by their general practitioner for a specialist opinion: a randomised trial. Lancet 2002;359: 1961-8.
Crits-Christoph P, Mintz J. Implications of therapist effects for the design and analysis of comparative studies of psychotherapies. J Consul Clin Psychol 1991;59: 20-6.
Roberts C. The implications of variation in outcome between health professionals for the design and analysis of randomised controlled trials. Stat Med 1999;18: 2605-15.
Greenfield S, Kaplan SH, Kahn R, Ninomiya J, Griffith JL. Profiling care provided by different groups of physicians: effects of patient case-mix (bias) and physician-level clustering on quality assessment results. Ann Intern Med 2002;136: 111-21.
Hoover DR. Clinical trials of behavioural interventions with heterogeneous teaching subgroup effects. Stat Med 2002;21: 1351-64.
Martindale C. The therapist as fixed effect fallacy in psychotherapy research. J Consult Clin Psychol 1978;46: 1526-30.
Paul SR,Donner A. Small sample performance of tests of homogeneity of odds ratios in k 2x2 tables. Stat Med 1992;11: 159-65.
Localio AR, Berlin JA, Ten Have TR, Kimmel SE. Adjustments for center in multicenter studies: an overview. Ann Intern Med 2001;135: 112-23.
Jones B, Teather D, Wang J, Lewis JA. A comparison of various estimators of a treatment difference for a multi-centre clinical trial. Stat Med 1998;17: 1767-77.
Senn S. Some controversies in planning and analysing multi-centre trials. Stat Med 1998;17: 1753-65.(Katherine J Lee, PhD student1, Simon G T)
Correspondence to: K J Lee kjl27@cam.ac.uk
Patient outcomes in many randomised trials depend crucially on the health professional delivering the intervention, but the resulting clustering is rarely considered in the analysis
Introduction
We reviewed all trials randomising individuals published in the BMJ during 2002, thus covering a wide range of medical areas. Trials randomising groups of individuals (cluster randomised trials) raise different issues and so were excluded.3 We recorded information on:
The type of clustering present—imposed or natural
Whether the issue was recognised (irrespective of whether it was accounted for in the statistical analysis)
How important we thought the clustering was, and
Whether we felt it had been adequately accounted for in the statistical analysis.
The final two items are subjective. Clustering judged as likely to be important was directly related to the outcomes in the trial (such as a therapist delivering an intervention). All multicentre trials were recorded as having natural clustering irrespective of the number of centres. In a validation exercise, a second reader independently reviewed a random sample of seven trials; there was 95% agreement on the items recorded.
We identified 42 trials (see bmj.com for list). Thirty eight had some form of clustering, with 17 (40%) having clustering by health professional imposed by the design of the trial (table 1). Only six out of the 38 trials mentioned clustering as a potential issue. Four of these allowed in some way for clustering in the analysis of the trial's results, although three of the four failed to recognise multiple sources of clustering.
Table 1 Assessment of clustering in 42 individually randomised trials published in BMJ in 2002
We classified clustering as unlikely to affect the results in 20 trials; four had no clustering (either single centre trials or trials where the intervention was delivered by a single health professional); and the remaining 16 were trials with natural clustering not directly related to the outcomes being assessed. Nineteen trials (45%) showed clustering that was likely to affect their results. Of these, only one attempted to take the issue into account. This trial, of community nurses specialising in Parkinson's disease,4 explicitly investigated the potential variability in results between nurses, although the researchers found it insignificant and disregarded it in the final analysis. We conclude that some potential for clustering exists in almost all trials, with over a third of trials having clustering by health professional imposed by the design. This clustering is infrequently acknowledged, and even more rarely adequately addressed.
Is clustering important?
Our analysis shows that potential clustering of outcomes is common in trials that randomise individuals but is usually ignored in the analysis of results. A review of trials in psychotherapy research also found that two thirds of trials ignored clustering.6 Clustering is most likely to have an important effect on results when health professionals actively deliver the intervention. Patients' outcomes may then depend crucially on the skill and enthusiasm of the professional involved. This may also be an issue in cluster randomised trials in which, although the clustering used in the randomisation process is recognised and generally adjusted for, further important forms of clustering may not be identified.
In the teleconsultation trial, the intervention effect became non-significant once clustering had been accounted for. Similar results have been reported in a trial comparing the delivery of minor acute care by the patient's general practitioner with that of commercial deputising services,7 in allowing for physician level clustering on the quality of diabetes care between specialty groups,8 and in studies of small group teaching.9
Summary points
Clustering is common in individually randomised trials
The potential effects of clustering are generally ignored in the analysis of trial results
Clustering is particularly important when interventions are delivered by more than one health professional
Ignoring clustering can lead to incorrect conclusions
Bigger sample sizes are needed to accommodate the potential for clustering
Clustering reduces the effective sample size, reducing the power of a trial to detect an intervention effect.7 10 Thus the best way to deal with the problem is to anticipate it at the time of design and to increase the sample size.9 We do not recommend trying to identify clustering on the basis of a statistical test of significance. Such tests lack power, and a nonsignificant result does not rule out the presence of important clustering.11 Rather, clustering should be anticipated on the basis of a trial's design, and so making allowance for it should follow as a consequence. Similar arguments apply both to cluster randomised trials3 and to individually randomised multicentre or international trials.12-14
Clustering also affects the generalisability of conclusions. For example, in therapy trials, the sample of therapists in the trial should be representative of those who are going to deliver the intervention in practice. Even if this is the case, the analysis must acknowledge the clusters for the conclusions to be justified.6 The issue of clustering of outcomes in randomised trials warrants much more attention than it has received so far, not only in design and analysis, but also in drawing justified and generalisable conclusions.
References for trials and details of clustering and the statistical analysis are on bmj.com
We thank the virtual outreach project team, in particular Paul Wallace and Julie Barber, for allowing us access to the data from the teleconsultation trial.
Contributors and sources: KJL is undertaking research on the topic of clustering in individually randomised trials. SGT has extensive experience in applied and methodological clinical trial research. SGT proposed the investigation. KJL reviewed the BMJ papers and reanalysed the teleconsultation trial. Both authors participated in drafting the article. KJL is guarantor.
Competing interests: None declared.
References
Dey P, Bundred N, Gibbs A, Hopwood P, Baildam A, Boggis C, et al. Costs and benefits of a one stop clinic compared with a dedicated breast clinic: randomised controlled trial. BMJ 2002;324: 507-11.
Koning S, Suijlekom-Smit LW, Nouwen JL, Verduin CM, Bernsen RM, Oranje AP, et al. Fusidic acid cream in the treatment of impetigo in general practice: double blind randomised placebo controlled trial. BMJ 2002;324: 203-6.
Donner A, Klar N. Design and analysis of cluster randomization trials in health research. London: Arnold, 2000.
Jarman B, Hurwitz B, Cook A, Bajekal M, Lee A. Effects of community based nurses specialising in Parkinson's disease on health outcome and costs: randomised controlled trial. BMJ 2002;324: 1072-5.
Wallace P, Haines A, Harrison R, Barber J, Thompson SG, Jacklin P, et al. Joint teleconsultations (virtual outreach) versus standard outpatient appointments for patients referred by their general practitioner for a specialist opinion: a randomised trial. Lancet 2002;359: 1961-8.
Crits-Christoph P, Mintz J. Implications of therapist effects for the design and analysis of comparative studies of psychotherapies. J Consul Clin Psychol 1991;59: 20-6.
Roberts C. The implications of variation in outcome between health professionals for the design and analysis of randomised controlled trials. Stat Med 1999;18: 2605-15.
Greenfield S, Kaplan SH, Kahn R, Ninomiya J, Griffith JL. Profiling care provided by different groups of physicians: effects of patient case-mix (bias) and physician-level clustering on quality assessment results. Ann Intern Med 2002;136: 111-21.
Hoover DR. Clinical trials of behavioural interventions with heterogeneous teaching subgroup effects. Stat Med 2002;21: 1351-64.
Martindale C. The therapist as fixed effect fallacy in psychotherapy research. J Consult Clin Psychol 1978;46: 1526-30.
Paul SR,Donner A. Small sample performance of tests of homogeneity of odds ratios in k 2x2 tables. Stat Med 1992;11: 159-65.
Localio AR, Berlin JA, Ten Have TR, Kimmel SE. Adjustments for center in multicenter studies: an overview. Ann Intern Med 2001;135: 112-23.
Jones B, Teather D, Wang J, Lewis JA. A comparison of various estimators of a treatment difference for a multi-centre clinical trial. Stat Med 1998;17: 1767-77.
Senn S. Some controversies in planning and analysing multi-centre trials. Stat Med 1998;17: 1753-65.(Katherine J Lee, PhD student1, Simon G T)