Do clinical trials improve quality of care A comparison of clinical processes and outcomes in patients in a clinical trial and similar patie
http://www.100md.com
《健康质量安全杂志》
1 Specialist Trainee in Public Health, Department of Clinical Epidemiology & Public Health, Bradford Teaching Hospitals NHS Foundation Trust, Bradford, UK
2 Consultant in Clinical Epidemiology and Public Health Medicine, Bradford Teaching Hospitals NHS Foundation Trust, Bradford, UK
3 Consultant Obstetrician and Gynaecologist, Bradford Teaching Hospitals NHS Foundation Trust, Bradford, UK
4 Regional Audit Midwife, Bradford Teaching Hospitals NHS Foundation Trust, Bradford, UK
5 Lecturer in Statistics, Nuffield Institute for Health, University of Leeds, Leeds, UK
ABSTRACT
Background: The conventional view that participants in randomised controlled trials sacrifice themselves for the good of future patients is challenged by increasing evidence to suggest that individual patients benefit from participation in trials.
Objective: To test the hypothesis that trial participants receive higher quality care and, as a consequence, have better outcomes than patients receiving guideline driven routine care.
Methods: Retrospective comparative study of 408 women with pre-eclampsia all managed according to a strict protocol. Trial participants were 86 women who participated in a multicentre randomised controlled trial of magnesium sulphate for the treatment of pre-eclampsia (Magpie Trial); 322 non-participants formed the control group. Indicators of the process of care and clinical outcomes were compared between the two groups.
Results: Trial participants were significantly more likely to have received daily blood tests (odds ratio (OR) 6.82, 95% CI 1.62 to 28.72) and had their respiration rate measured hourly (OR 3.42, 95% CI 1.69 to 6.92) than control patients. There were no significant differences in other markers of clinical process and no significant difference in clinical outcomes.
Conclusion: This study shows minor differences in process markers and no difference in clinical outcomes between patients in a clinical trial and patients receiving protocol driven care. The benefits of improved clinical care that have previously been associated with being in a trial may be explained by the use of clear clinical protocols. In routine practice, patients may be well advised to insist on treatment as part of a protocol.
Keywords: randomised controlled trials; quality of care; pre-eclampsia
The recognition that health care should be based on reliable and up to date evidence has drawn attention to the need for high quality randomised trials addressing important and previously unanswered questions.1 Clinical trials are widely acknowledged as the design of choice for evaluating medical and surgical treatments and are essential for clinical progress.2,3 However, in terms of their status and success, there are two very different perceptions of clinical trials. Firstly, there is a conventional wisdom that patients who participate in randomised controlled trials are subject to experimentation and increased risk and this ideology underpins the public perception of sacrifice and altruism. To some extent this is understandable. Well conducted clinical trials are clearly beneficial to society in the long term, and the generation of clinical trial evidence is crucial for efforts to reduce morbidity and mortality and improve the health of populations.4 However, individual patients instinctively hope to derive some personal benefit from whichever treatment they are given as part of their health care and, as such, are unlikely to opt for a process that they see as sacrificial rather than beneficial. The proportion of patients consenting to enter clinical trials in maternity care therefore remains small.5
Conversely, there is a professional perception that clinical trials represent superior care.6 Indeed, Iain Chalmers, co-founder of the Cochrane Collaboration, has repeatedly voiced his preference to enter a trial rather than receive inadequately evaluated routine care (Chalmers I, Report of a Seminar organised by Consumers in NHS Research and MRC Clinical Trials Unit, 2000). In contrast to the public perception, many health professionals recognise that clinical trials are good for us as a society and support the concept of a beneficial trial effect for participants.6,7 Perhaps unfairly, health professionals have the upper hand in that they have access and exposure to current evidence, the bulk of which seems to support a favourable trial effect.8,9 Indeed, a number of studies have identified short term beneficial effects to individual trial participants.10–12
On the whole, previous research has focused on identifying the effect rather than the cause, but a number of potential sources of a trial effect have been reported.8 These include the Hawthorne effect whereby merely the process of being observed leads to benefit,13 the treatment effect resulting from the treatment under study being superior to the standard alternative, the care effect where the trial protocol requires more comprehensive care,8 and the protocol effect. The protocol effect is a simple but potentially significant concept. Clinical guidelines improve the quality of clinical practice,14 so it is clearly plausible that a similar effect can be achieved by the systemised protocols used in clinical trials that guide clinicians through clear and explicit processes of care, resulting in improved outcomes in trial participants whether in the intervention or the control group. In order to measure a protocol effect, it would be feasible to conduct a controlled trial in which participants were randomly allocated entry to the clinical trial or care following a standardised guideline or protocol. However, clinical trials are not set up to be so introspective and have clear aims in evaluating the effect of the intervention under scrutiny.
The nesting of a clinical trial within the context of care being provided according to an explicit protocol based guideline provided the opportunity for a natural comparison. The aim of this study was to observe a naturally occurring comparison to ascertain whether there is any clinical benefit to enrolling in a clinical trial when the potential advantage of a protocol effect is removed.
METHODS
Sixteen units throughout the Yorkshire region had been managing women with severe pre-eclampsia according to a strict management protocol as part of evidence based regional pre-eclampsia guidelines introduced in 1997 by the Yorkshire Obstetric Critical Care Group (unpublished). The guidelines are reviewed annually by the group and are accepted as best practice. A continuous audit of the guidelines began in 1997.
From July 1998 until November 2001 six of the 16 units were recruited to a multicentre randomised controlled trial of magnesium sulphate for the treatment of pre-eclampsia (Magpie Trial). Trial participants were randomised to receive either magnesium sulphate or a placebo of normal saline. Both those participating in the trial and those who did not continued to be managed using the regional guidelines. Hence, subsequent treatment and management were identical, the only difference being that magnesium sulphate was randomly allocated to some trial participants. As the Magpie Trial Collaborative Group was continuing to analyse Magpie study data for further related research, we were unable to unblind treatment allocation for trial participants to determine whether treatment or placebo was given.
This comparative study seized on a unique opportunity to use the routinely collected continuous audit data to compare women participating in the trial with those who did not participate.
Audit data were recorded by a network of midwives based at the participating units. Data were then collected on site by a regional audit midwife and entered into a central Access database. Incomplete data were retrieved through further site visits by the regional audit midwife, ensuring an accurate and complete data set. The reported blood tests and heparin therapy (thromboprophylaxis) were requirements of the regional guidelines, not of the Magpie trial which required no additional tests or monitoring. Indicators of the process of care (table 1) and clinical outcomes (tables 2 and 3) were compared between the two groups.
Analysis of data
Odds ratios with 95% confidence intervals were calculated for all dichotomous data using EpiInfo 2002 and confidence interval analysis (CIA) software. Where the odds ratio was inappropriate, Fisher’s exact test was used. Means, standard deviations, and mean differences with 95% confidence intervals were calculated for continuous data, again using CIA software. The null hypothesis was that there was no difference in process or outcome measures.
Multiple tests (n = 10) were applied to the process outcomes. Spurious positive results may occur if no correction is used. We applied the most conservative adjustment using the Bonferroni correction which assumes all 10 tests were independent.
RESULTS
Eighty six women from the six trial sites entered the trial (fig 1). A further 322 women from the same trial sites did not enter Magpie but were managed according to the guidelines during the recruitment period for the trial and formed the comparison group. Table 1 shows the comparison of indicators of clinical process between the two groups. There were statistically significant differences in daily blood tests (odds ratio (OR) 6.82, 95% CI 1.62 to 28.72), recording of hourly respirations (OR 3.42, 95% CI 1.69 to 6.92), and thromboprophylaxis (OR 2.21, 95% CI 1.22 to 3.98). The Bonferroni correction uses a significance level of 0.0051. Even with this conservative adjustment, hourly respirations recorded remained significant. Daily blood tests and thromboprophylaxis were close to significance despite the great conservativeness of the correction. There were no significant differences in the other seven process indicators.
Clinical outcomes are shown in tables 2 and 3. Birth weight (mean difference 174.0, 95% CI –97.1 to 445.1) and length of time on the special care baby unit (mean difference 2.57, 95% CI –1.93 to 7.07) were similar in both groups with no significant differences. There was no significant difference in admission to intensive care units (OR 3.81, 95% CI 0.53 to 27.44), incidence of pulmonary oedema (OR 0.93, 95% CI 0.29 to 4.48), and length of time spent on the postnatal guidelines (mean difference 2.45, 95% CI –1.54 to 6.44). Three women from the control group experienced eclamptic fits compared with none in the trial group (p = 0.39). The Magpie trial found that magnesium sulphate when used to treat pre-eclampsia halves the risk of eclampsia.15
DISCUSSION
The results of this study suggest that compliance with clinical processes is similar for trial participants and non-participants in terms of observations, recording, and clinician involvement. There are, however, two exceptions. Routine daily blood tests and respiration recordings were requirements of the clinical guidelines and should therefore have been performed for all women in both groups. However, women in the trial were more likely to undergo daily blood tests and respiration recordings, suggesting some benefit of trial participation and supporting a positive trial effect in process markers. This may be related to the trial or it may be a chance finding. The sample is small, particularly in relation to hourly respiration recordings in the comparison group. Hourly respiratory rate monitoring was required for all women receiving or potentially receiving magnesium sulphate. This included all women in the trial group and 65 women in the comparison group who were given magnesium sulphate. Despite the differences in measures of process, there were no significant differences in clinical outcomes between the two groups.
This study benefited from robust clinical audit data that enabled a unique comparison of trial participants and non-participants within a protocol environment. Although the eligibility criteria for the regional pre-eclampsia guidelines and the Magpie trial were similar, it cannot be assumed that all the women in the comparison group were eligible for trial entry. No recruitment rate is available for the Magpie trial and only one of the six units participating in the trial from the Yorkshire region recorded trial eligibility and recruitment. Furthermore, it was not possible from the data available to assess severity of pre-eclampsia and this could have influenced selection of potential trial entrants and their subsequent management. There is potential for a Hawthorne effect in this study as clinicians were aware of the ongoing audit of the pre-eclampsia guidelines; however, as all women in both groups were included in the audit, any effect would be equally distributed across both groups. This study focused on an obstetric trial but we have no reason to believe that it is fundamentally different from other clinical trials or doubt the generalisability of our findings.
Previous studies comparing patients in trials with those receiving routine care generally report improved primary outcomes for trial participants.10,11,16–18 Only two studies report similar outcomes19,20 and there is no evidence to suggest that outcomes are worsened by trial participation. This study does not confirm any clear benefit to trial participation. However, it differs from previous studies in two ways. Firstly, all the patients in this study were managed according to a clear and explicit clinical protocol. In a trial environment the protocol ensures that care is consistent and based on available evidence, and strict trial protocols encourage detail and compliance which is known to result in better care.21 Clinical guidelines, care pathways and protocols have the potential to simulate the beneficial effects of a trial protocol and there is strong evidence that care significantly improves following their introduction.14,22 In this study potential differences between trial participants and non-participants may have been diluted by the presence of a protocol. Both groups received care according to the same clinical protocol. As such, the comparison group benefited from enhanced standardisation of care and experienced similar outcomes to the trial participants. This suggests that, where previously benefits to trial participation have been identified, they may be due to the presence of a protocol within the trial and the absence of a protocol outside the trial.
Key messages
Current evidence suggests that patients may do better in a clinical trial.
The benefit may derive from the systemised protocols used in clinical trials.
Effectively implemented clinical protocols have the potential to simulate the beneficial effects of a trial protocol.
Secondly, our study is one of only a few studies evaluating the trial effect that compares measures of clinical process. Measures of process of care are generally more sensitive indicators of quality of care than outcomes.23 Most bad outcomes cannot be prevented or are side effects of treatment.24 It may therefore be appropriate to concentrate on process measures when assessing quality of care.
In terms of clinical practice, current evidence suggests that patients are likely to do better in a trial and recruitment should be encouraged. In practice there will never be an appropriate clinical trial for every patient. However, care provided as part of a clear and effectively implemented protocol may enhance standardisation of care and all patients may experience better care if clinical guidelines and protocols were more widely incorporated into routine practice.
Further research is necessary to develop these findings and confirm the existence and nature of a protocol effect. Not all patients have the opportunity to take part in a trial but, perhaps, patients should be encouraged to ask if their treatment is being given as part of an agreed protocol and, if not, to ask why not.
ACKNOWLEDGEMENTS
The authors thank the YOCCG link midwives for their complete and accurate data, and Dee Kyle and Simon Gates for earlier advice and comments.
FOOTNOTES
Conflict of interest: none
REFERENCES
Gulmezoglu M, Hofmeyer J, Duley L, et al. Randomised controlled trials in maternal and perinatal medicine: global partnerships are the way forward. Br J Obstet Gynaecol 1998;105:1244–7.
Featherstone K, Dononvan JL. Random allocation or allocation at random Patient perspectives of participation in a RCT. BMJ 1998;317:1177–80.
Brocklehurst P . Randomised controlled trials in perinatal medicine: recruitment of a pregnant woman or her newborn child to more than one trial. Br J Obstet Gynaecol 1997;104:765–7.
Collins R, Peto R, Gray R, et al. Large scale randomised evidence in trials and overviews. In: Weatherill DJ, Ledingham JGG, Warrell DA, eds. Oxford textbook of public health medicine. Oxford: Oxford University Press, 1996.
West J, Tuffnell DJ, Jankowicz D, et al. Randomised controlled trials: changing the culture of maternity care. Br J Midwifery 2001;9:766–9.
Chalmers I, Lindley RI. Double standards on informed consent to treatment. In: Doyal L, Tobias JS, eds. Informed consent in medical research. London: BMJ Books, 2001.
Lees R . If I had a stroke.... Lancet 1998;352 (Suppl III) :28–30.
Braunholtz DA, Edwards SLJ, Lilford R. Are randomised trials good for us Evidence for a trial effect. J Clin Epidemiol 2001;54:217–24.
Edwards SLJ, Lilford RJ, Braunholtz DA, et al. Ethical issues in the design and conduct of RCTs. Health Technol Assess 1998;2 (15).
Hallstrom A, Friedman L, Denes P, et al. Do arrhythmia patients improve survival by participating in randomised controlled trials Observations from the CAST trial and AVID trial. Controlled Clin Trials 2003;24:341–52.
Schmidt B, Gillie P, Caco C, et al. Do sick newborn infants benefit from participation in a RCT J Pediatr 1999;134:151–5.
Weijer C, Friedman B, Fuks A, et al. What difference does it make to be treated in a clinical trial A pilot study. Clin Invest Med 1996;19:179–83.
De Amici D, Klersy C, Ramajoli F, et al. Impact of the Hawthorne effect in a longitudinal clinical study. The case for anaesthesia. Controlled Clin Trials 2000;21:103–14.
Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice. A systematic review of rigorous evaluations. Lancet 1993;342:1317–22.
Magpie Trial Collaborative Group. Do women with pre-eclampsia, and their babies, benefit from magnesium sulphate The Magpie Trial: a randomised placebo controlled trial, Lancet 2002;359:1877–90.
Vist GE, Hagen KB, Devereaux PJ, et al. Outcomes of patients who participate in RCTs versus those of similar patients who do not participate (protocol for a Cochrane Review). In: The Cochrane Library. Issue 3. Oxford: Update Software, 2003.
Karjalainen S, Palva I. Do treatment protocols improve end results A study of survival of patients with multiple myeloma in Finland. BMJ 1989;299:1069–72.
Jha P, Deboer D, Sykora K, et al. Characteristics and mortality outcomes of thrombolysis trial participants: a population based comparison. J Am Coll Cardiol 1996;27:1335–42.
Coronary Artery Surgery Study (CASS). A randomised trial of coronary artery bypass surgery (CASS). Comparability of entry characteristics and survival in randomised patients and non-randomised patients meeting randomisation criteria. J Am Coll Cardiol 1984;3:114–28.
Bertelson K . Protocol allocation and exclusion in two Danish randomised trials in ovarian cancer. Br J Cancer 1991;64:1172–6.
Lilford RJ, Thornton JG, Braunholtz D. Clinical trials and rare diseases: a way out of the conundrum. BMJ 1995;311:1621–5.
Campbell H, Hotchkiss R, Bradshaw N, et al. Integrated care pathways. BMJ 1998;316:133–7.
Mant J, Hicks N. Detecting differences in quality of care: the sensitivity of measures of process and outcome in treating acute myocardial infarction. BMJ 1995;311:793–6.
Lilford RJ, Mohammed M, Braunholtz D, et al. The measurement of active cross-methodological issues. Qual Saf Health Care 2003;12 (Suppl II) :ii8–12.(J West, J Wright, D Tuffn)
2 Consultant in Clinical Epidemiology and Public Health Medicine, Bradford Teaching Hospitals NHS Foundation Trust, Bradford, UK
3 Consultant Obstetrician and Gynaecologist, Bradford Teaching Hospitals NHS Foundation Trust, Bradford, UK
4 Regional Audit Midwife, Bradford Teaching Hospitals NHS Foundation Trust, Bradford, UK
5 Lecturer in Statistics, Nuffield Institute for Health, University of Leeds, Leeds, UK
ABSTRACT
Background: The conventional view that participants in randomised controlled trials sacrifice themselves for the good of future patients is challenged by increasing evidence to suggest that individual patients benefit from participation in trials.
Objective: To test the hypothesis that trial participants receive higher quality care and, as a consequence, have better outcomes than patients receiving guideline driven routine care.
Methods: Retrospective comparative study of 408 women with pre-eclampsia all managed according to a strict protocol. Trial participants were 86 women who participated in a multicentre randomised controlled trial of magnesium sulphate for the treatment of pre-eclampsia (Magpie Trial); 322 non-participants formed the control group. Indicators of the process of care and clinical outcomes were compared between the two groups.
Results: Trial participants were significantly more likely to have received daily blood tests (odds ratio (OR) 6.82, 95% CI 1.62 to 28.72) and had their respiration rate measured hourly (OR 3.42, 95% CI 1.69 to 6.92) than control patients. There were no significant differences in other markers of clinical process and no significant difference in clinical outcomes.
Conclusion: This study shows minor differences in process markers and no difference in clinical outcomes between patients in a clinical trial and patients receiving protocol driven care. The benefits of improved clinical care that have previously been associated with being in a trial may be explained by the use of clear clinical protocols. In routine practice, patients may be well advised to insist on treatment as part of a protocol.
Keywords: randomised controlled trials; quality of care; pre-eclampsia
The recognition that health care should be based on reliable and up to date evidence has drawn attention to the need for high quality randomised trials addressing important and previously unanswered questions.1 Clinical trials are widely acknowledged as the design of choice for evaluating medical and surgical treatments and are essential for clinical progress.2,3 However, in terms of their status and success, there are two very different perceptions of clinical trials. Firstly, there is a conventional wisdom that patients who participate in randomised controlled trials are subject to experimentation and increased risk and this ideology underpins the public perception of sacrifice and altruism. To some extent this is understandable. Well conducted clinical trials are clearly beneficial to society in the long term, and the generation of clinical trial evidence is crucial for efforts to reduce morbidity and mortality and improve the health of populations.4 However, individual patients instinctively hope to derive some personal benefit from whichever treatment they are given as part of their health care and, as such, are unlikely to opt for a process that they see as sacrificial rather than beneficial. The proportion of patients consenting to enter clinical trials in maternity care therefore remains small.5
Conversely, there is a professional perception that clinical trials represent superior care.6 Indeed, Iain Chalmers, co-founder of the Cochrane Collaboration, has repeatedly voiced his preference to enter a trial rather than receive inadequately evaluated routine care (Chalmers I, Report of a Seminar organised by Consumers in NHS Research and MRC Clinical Trials Unit, 2000). In contrast to the public perception, many health professionals recognise that clinical trials are good for us as a society and support the concept of a beneficial trial effect for participants.6,7 Perhaps unfairly, health professionals have the upper hand in that they have access and exposure to current evidence, the bulk of which seems to support a favourable trial effect.8,9 Indeed, a number of studies have identified short term beneficial effects to individual trial participants.10–12
On the whole, previous research has focused on identifying the effect rather than the cause, but a number of potential sources of a trial effect have been reported.8 These include the Hawthorne effect whereby merely the process of being observed leads to benefit,13 the treatment effect resulting from the treatment under study being superior to the standard alternative, the care effect where the trial protocol requires more comprehensive care,8 and the protocol effect. The protocol effect is a simple but potentially significant concept. Clinical guidelines improve the quality of clinical practice,14 so it is clearly plausible that a similar effect can be achieved by the systemised protocols used in clinical trials that guide clinicians through clear and explicit processes of care, resulting in improved outcomes in trial participants whether in the intervention or the control group. In order to measure a protocol effect, it would be feasible to conduct a controlled trial in which participants were randomly allocated entry to the clinical trial or care following a standardised guideline or protocol. However, clinical trials are not set up to be so introspective and have clear aims in evaluating the effect of the intervention under scrutiny.
The nesting of a clinical trial within the context of care being provided according to an explicit protocol based guideline provided the opportunity for a natural comparison. The aim of this study was to observe a naturally occurring comparison to ascertain whether there is any clinical benefit to enrolling in a clinical trial when the potential advantage of a protocol effect is removed.
METHODS
Sixteen units throughout the Yorkshire region had been managing women with severe pre-eclampsia according to a strict management protocol as part of evidence based regional pre-eclampsia guidelines introduced in 1997 by the Yorkshire Obstetric Critical Care Group (unpublished). The guidelines are reviewed annually by the group and are accepted as best practice. A continuous audit of the guidelines began in 1997.
From July 1998 until November 2001 six of the 16 units were recruited to a multicentre randomised controlled trial of magnesium sulphate for the treatment of pre-eclampsia (Magpie Trial). Trial participants were randomised to receive either magnesium sulphate or a placebo of normal saline. Both those participating in the trial and those who did not continued to be managed using the regional guidelines. Hence, subsequent treatment and management were identical, the only difference being that magnesium sulphate was randomly allocated to some trial participants. As the Magpie Trial Collaborative Group was continuing to analyse Magpie study data for further related research, we were unable to unblind treatment allocation for trial participants to determine whether treatment or placebo was given.
This comparative study seized on a unique opportunity to use the routinely collected continuous audit data to compare women participating in the trial with those who did not participate.
Audit data were recorded by a network of midwives based at the participating units. Data were then collected on site by a regional audit midwife and entered into a central Access database. Incomplete data were retrieved through further site visits by the regional audit midwife, ensuring an accurate and complete data set. The reported blood tests and heparin therapy (thromboprophylaxis) were requirements of the regional guidelines, not of the Magpie trial which required no additional tests or monitoring. Indicators of the process of care (table 1) and clinical outcomes (tables 2 and 3) were compared between the two groups.
Analysis of data
Odds ratios with 95% confidence intervals were calculated for all dichotomous data using EpiInfo 2002 and confidence interval analysis (CIA) software. Where the odds ratio was inappropriate, Fisher’s exact test was used. Means, standard deviations, and mean differences with 95% confidence intervals were calculated for continuous data, again using CIA software. The null hypothesis was that there was no difference in process or outcome measures.
Multiple tests (n = 10) were applied to the process outcomes. Spurious positive results may occur if no correction is used. We applied the most conservative adjustment using the Bonferroni correction which assumes all 10 tests were independent.
RESULTS
Eighty six women from the six trial sites entered the trial (fig 1). A further 322 women from the same trial sites did not enter Magpie but were managed according to the guidelines during the recruitment period for the trial and formed the comparison group. Table 1 shows the comparison of indicators of clinical process between the two groups. There were statistically significant differences in daily blood tests (odds ratio (OR) 6.82, 95% CI 1.62 to 28.72), recording of hourly respirations (OR 3.42, 95% CI 1.69 to 6.92), and thromboprophylaxis (OR 2.21, 95% CI 1.22 to 3.98). The Bonferroni correction uses a significance level of 0.0051. Even with this conservative adjustment, hourly respirations recorded remained significant. Daily blood tests and thromboprophylaxis were close to significance despite the great conservativeness of the correction. There were no significant differences in the other seven process indicators.
Clinical outcomes are shown in tables 2 and 3. Birth weight (mean difference 174.0, 95% CI –97.1 to 445.1) and length of time on the special care baby unit (mean difference 2.57, 95% CI –1.93 to 7.07) were similar in both groups with no significant differences. There was no significant difference in admission to intensive care units (OR 3.81, 95% CI 0.53 to 27.44), incidence of pulmonary oedema (OR 0.93, 95% CI 0.29 to 4.48), and length of time spent on the postnatal guidelines (mean difference 2.45, 95% CI –1.54 to 6.44). Three women from the control group experienced eclamptic fits compared with none in the trial group (p = 0.39). The Magpie trial found that magnesium sulphate when used to treat pre-eclampsia halves the risk of eclampsia.15
DISCUSSION
The results of this study suggest that compliance with clinical processes is similar for trial participants and non-participants in terms of observations, recording, and clinician involvement. There are, however, two exceptions. Routine daily blood tests and respiration recordings were requirements of the clinical guidelines and should therefore have been performed for all women in both groups. However, women in the trial were more likely to undergo daily blood tests and respiration recordings, suggesting some benefit of trial participation and supporting a positive trial effect in process markers. This may be related to the trial or it may be a chance finding. The sample is small, particularly in relation to hourly respiration recordings in the comparison group. Hourly respiratory rate monitoring was required for all women receiving or potentially receiving magnesium sulphate. This included all women in the trial group and 65 women in the comparison group who were given magnesium sulphate. Despite the differences in measures of process, there were no significant differences in clinical outcomes between the two groups.
This study benefited from robust clinical audit data that enabled a unique comparison of trial participants and non-participants within a protocol environment. Although the eligibility criteria for the regional pre-eclampsia guidelines and the Magpie trial were similar, it cannot be assumed that all the women in the comparison group were eligible for trial entry. No recruitment rate is available for the Magpie trial and only one of the six units participating in the trial from the Yorkshire region recorded trial eligibility and recruitment. Furthermore, it was not possible from the data available to assess severity of pre-eclampsia and this could have influenced selection of potential trial entrants and their subsequent management. There is potential for a Hawthorne effect in this study as clinicians were aware of the ongoing audit of the pre-eclampsia guidelines; however, as all women in both groups were included in the audit, any effect would be equally distributed across both groups. This study focused on an obstetric trial but we have no reason to believe that it is fundamentally different from other clinical trials or doubt the generalisability of our findings.
Previous studies comparing patients in trials with those receiving routine care generally report improved primary outcomes for trial participants.10,11,16–18 Only two studies report similar outcomes19,20 and there is no evidence to suggest that outcomes are worsened by trial participation. This study does not confirm any clear benefit to trial participation. However, it differs from previous studies in two ways. Firstly, all the patients in this study were managed according to a clear and explicit clinical protocol. In a trial environment the protocol ensures that care is consistent and based on available evidence, and strict trial protocols encourage detail and compliance which is known to result in better care.21 Clinical guidelines, care pathways and protocols have the potential to simulate the beneficial effects of a trial protocol and there is strong evidence that care significantly improves following their introduction.14,22 In this study potential differences between trial participants and non-participants may have been diluted by the presence of a protocol. Both groups received care according to the same clinical protocol. As such, the comparison group benefited from enhanced standardisation of care and experienced similar outcomes to the trial participants. This suggests that, where previously benefits to trial participation have been identified, they may be due to the presence of a protocol within the trial and the absence of a protocol outside the trial.
Key messages
Current evidence suggests that patients may do better in a clinical trial.
The benefit may derive from the systemised protocols used in clinical trials.
Effectively implemented clinical protocols have the potential to simulate the beneficial effects of a trial protocol.
Secondly, our study is one of only a few studies evaluating the trial effect that compares measures of clinical process. Measures of process of care are generally more sensitive indicators of quality of care than outcomes.23 Most bad outcomes cannot be prevented or are side effects of treatment.24 It may therefore be appropriate to concentrate on process measures when assessing quality of care.
In terms of clinical practice, current evidence suggests that patients are likely to do better in a trial and recruitment should be encouraged. In practice there will never be an appropriate clinical trial for every patient. However, care provided as part of a clear and effectively implemented protocol may enhance standardisation of care and all patients may experience better care if clinical guidelines and protocols were more widely incorporated into routine practice.
Further research is necessary to develop these findings and confirm the existence and nature of a protocol effect. Not all patients have the opportunity to take part in a trial but, perhaps, patients should be encouraged to ask if their treatment is being given as part of an agreed protocol and, if not, to ask why not.
ACKNOWLEDGEMENTS
The authors thank the YOCCG link midwives for their complete and accurate data, and Dee Kyle and Simon Gates for earlier advice and comments.
FOOTNOTES
Conflict of interest: none
REFERENCES
Gulmezoglu M, Hofmeyer J, Duley L, et al. Randomised controlled trials in maternal and perinatal medicine: global partnerships are the way forward. Br J Obstet Gynaecol 1998;105:1244–7.
Featherstone K, Dononvan JL. Random allocation or allocation at random Patient perspectives of participation in a RCT. BMJ 1998;317:1177–80.
Brocklehurst P . Randomised controlled trials in perinatal medicine: recruitment of a pregnant woman or her newborn child to more than one trial. Br J Obstet Gynaecol 1997;104:765–7.
Collins R, Peto R, Gray R, et al. Large scale randomised evidence in trials and overviews. In: Weatherill DJ, Ledingham JGG, Warrell DA, eds. Oxford textbook of public health medicine. Oxford: Oxford University Press, 1996.
West J, Tuffnell DJ, Jankowicz D, et al. Randomised controlled trials: changing the culture of maternity care. Br J Midwifery 2001;9:766–9.
Chalmers I, Lindley RI. Double standards on informed consent to treatment. In: Doyal L, Tobias JS, eds. Informed consent in medical research. London: BMJ Books, 2001.
Lees R . If I had a stroke.... Lancet 1998;352 (Suppl III) :28–30.
Braunholtz DA, Edwards SLJ, Lilford R. Are randomised trials good for us Evidence for a trial effect. J Clin Epidemiol 2001;54:217–24.
Edwards SLJ, Lilford RJ, Braunholtz DA, et al. Ethical issues in the design and conduct of RCTs. Health Technol Assess 1998;2 (15).
Hallstrom A, Friedman L, Denes P, et al. Do arrhythmia patients improve survival by participating in randomised controlled trials Observations from the CAST trial and AVID trial. Controlled Clin Trials 2003;24:341–52.
Schmidt B, Gillie P, Caco C, et al. Do sick newborn infants benefit from participation in a RCT J Pediatr 1999;134:151–5.
Weijer C, Friedman B, Fuks A, et al. What difference does it make to be treated in a clinical trial A pilot study. Clin Invest Med 1996;19:179–83.
De Amici D, Klersy C, Ramajoli F, et al. Impact of the Hawthorne effect in a longitudinal clinical study. The case for anaesthesia. Controlled Clin Trials 2000;21:103–14.
Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice. A systematic review of rigorous evaluations. Lancet 1993;342:1317–22.
Magpie Trial Collaborative Group. Do women with pre-eclampsia, and their babies, benefit from magnesium sulphate The Magpie Trial: a randomised placebo controlled trial, Lancet 2002;359:1877–90.
Vist GE, Hagen KB, Devereaux PJ, et al. Outcomes of patients who participate in RCTs versus those of similar patients who do not participate (protocol for a Cochrane Review). In: The Cochrane Library. Issue 3. Oxford: Update Software, 2003.
Karjalainen S, Palva I. Do treatment protocols improve end results A study of survival of patients with multiple myeloma in Finland. BMJ 1989;299:1069–72.
Jha P, Deboer D, Sykora K, et al. Characteristics and mortality outcomes of thrombolysis trial participants: a population based comparison. J Am Coll Cardiol 1996;27:1335–42.
Coronary Artery Surgery Study (CASS). A randomised trial of coronary artery bypass surgery (CASS). Comparability of entry characteristics and survival in randomised patients and non-randomised patients meeting randomisation criteria. J Am Coll Cardiol 1984;3:114–28.
Bertelson K . Protocol allocation and exclusion in two Danish randomised trials in ovarian cancer. Br J Cancer 1991;64:1172–6.
Lilford RJ, Thornton JG, Braunholtz D. Clinical trials and rare diseases: a way out of the conundrum. BMJ 1995;311:1621–5.
Campbell H, Hotchkiss R, Bradshaw N, et al. Integrated care pathways. BMJ 1998;316:133–7.
Mant J, Hicks N. Detecting differences in quality of care: the sensitivity of measures of process and outcome in treating acute myocardial infarction. BMJ 1995;311:793–6.
Lilford RJ, Mohammed M, Braunholtz D, et al. The measurement of active cross-methodological issues. Qual Saf Health Care 2003;12 (Suppl II) :ii8–12.(J West, J Wright, D Tuffn)