Assessing ethics of trials in systematic reviews
http://www.100md.com
《英国医生杂志》
1 Department of Family Medicine, Tel-Aviv University, Tel-Aviv, Israel, 2 Internal Medicine and Infectious Diseases Unit, Beilinson Hospital, Rabin Medical Centre, Petah Tikva, Israel, 3 Beilinson Campus, Rabin Medical Centre
Correspondence to: M A Weingarten weingml@post.tau.ac.il
Routine assessment may improve ethical standards and overall quality of trials
Our awareness of the requirements for ethical clinical research has increased over the past century. Research ethics committees were set up after the Declaration of Helsinki to review research proposals. Many journals now require a statement that ethical approval has been obtained before they consider a research report for publication. Nevertheless, many published studies do not come up to standard, or at least do not report that they do. For example, 30 out of 37 consecutive studies published in five general paediatric journals did not report whether informed consent was obtained. Twenty four of them did not report whether the committee on research ethics had approved the study.1 We propose that systematic reviews of experimental clinical research on humans should also include information on the ethical standards of the trials.
Why include ethical information?
The main reason for including ethics in the checklist of systematic reviews is to increase awareness in the scientific community about the need for high ethical standards in research on humans. The proposal would also encourage reviewers to identify those occasional studies that were so unethical that there may be doubts about the morality of using the results. Although such trials are rare, history has given us too many real examples to allow us to be complacent.2 3 Opinions differ on whether it is justified to disseminate the results of such studies.4 Either way, a conscious decision should be made and revealed to readers of the review.
Issues around ethical quality overlap importantly with the central issues of the validity, reliability, and generalisability of research findings. They relate to some of the more subtle potential sources of bias in experimental clinical research. It is thus important to include ethical assessment in systematic reviews on prudential grounds as well as on moral grounds. The results of ethically sound trials may be more reliable than those from other trials. In a review of 767 randomised controlled trials published during 1993-5 in the New England Journal of Medicine, Lancet, BMJ, and JAMA, trials of higher methodological quality were more likely to provide information about their ethical aspects; so it seems that more reliable research is also more sensitive to the ethical requirements of research.5
Which issues should we assess?
Ethical issues at the level of a systematic review are different from those for original research, and it might be unfair to examine all trials under the same ethical loupe. Older trials will have been done when ethical standards of research had not been as clearly formulated as they are today. Ethical standards depend not only on the time the study was performed but also on place and social mores.
Nevertheless, we think that examination of the ethical standards of trials should form part of every review, but the ethical principles to be applied need to be defined. Among the approaches to research ethics, we find that proposed by Foster most attractive. She suggests examining the project from three perspectives: goals, duties, and rights.6 We used this model to devise a protocol to help assess the ethical quality of clinical trials included in systematic reviews (box).
Should systematic reviews merely delineate the ethical quality or attempt to integrate the observations into its conclusions? We believe that reviews should at least include a report of the ethical assessment and that the implications for further research should address the ethical gaps observed in existing studies.
How would the protocol work in practice?
We applied our protocol to a recently published systematic review of trials comparing lactam monotherapy versus lactam combined with aminoglycoside for febrile neutropenia.9 The review included 47 studies carried out during 1981-2000 and 7807 randomised patients.
Goal related considerations
Financial support—Twenty eight trials gave information on financial support: all had received funding from the company that manufactured the monotherapy antibiotic. Three trials declared research grant support as well as commercial support.
Conflicts of interest—None of the studies included a statement related to potential conflicts of interest.
Justification—Thirty seven studies did not report a sample size calculation. We calculated that an equivalence trial design with an overall success rate of 60% (as observed in this review), and permitting a 20% difference between the treatments, would require around 200 assessable patients to ensure an 80% chance of rejecting at = 0.05 the null hypothesis of equivalence.10 Thirty eight trials had fewer than 200 patients. This requirement is modest given the broad difference permitted between the treatments. Compilation of all existing data in this review led us to believe that further trials comparing lactam monotherapy with a narrower spectrum lactam combined with an aminoglycoside would be unjustified. However, we could not identify a specific time at which this conclusion should have been reached.
Guide for ethical assessment of trials in systematic reviews
Goal related considerations
Is there a clear declaration on financial support in all trials?
Is there a statement that relates to potential conflicts of interest in all trials?
Justification—Could the results have been obtained by laboratory or animal experiments? Were any of the trials superfluous? Was the size of the study sufficient to achieve adequate statistical power?
Publication bias—How many of the identified trials remained unpublished? Is bias detectable by funnel plot analysis?
Duty related considerations
? Were the comparators appropriate? If a placebo was used, was it justified?
Rights related considerations
Safety—Was the risk for participants appropriate to the importance of the research?7 Was appropriate follow up care assured?
Was informed consent obtained?
When participants had reduced competence, were appropriate measures taken to protect their best interests?8
Were adequate steps taken to prevent unauthorised access to personal and clinical data?
Global considerations
? Was the study approved by a research ethics committee?
Publication bias—Five trials completed before 1990 (943 patients) remained unpublished or were published only as conference proceedings. The results of one study including 460 patients were never formally published.11 Although we included these studies in our review, the funnel plot showed that small studies favouring combination therapy might be missing.
Duty related considerations
Appropriate comparators—We considered antibiotic combinations advised by guidelines contemporary to the study years as adequate.12-14 Twenty trials used an inadequate comparator on this basis.
Rights related considerations
Safety—We considered monitoring aminoglycoside serum concentrations at any time during the study and creatinine at least twice weekly as minimal follow up measures to ensure patients' safety. Thirty one trials did not report aminoglycoside monitoring and 17 had inadequate creatinine follow up.
Informed consent—Twenty nine trials reported that informed consent had been obtained from adult patients.
Participants of reduced competence—Twenty five trials included patients younger than 16 years old, and in only five of these was consent obtained from parents or relatives. The trials covered a variety of patient populations, including some seriously ill patients with disseminated carcinoma and geriatric patients. Consent was sometimes given by the family rather than the patient, and in those cases it was not clear whether the family had any power of attorney to act on the patient's behalf.
Confidentiality—There was no information on steps taken to ensure confidentiality in the published reports.
Global considerations
Approval by research ethics committee—Twenty two studies stated that they had been approved by a research ethics committee. We noted an association between this approval and other ethical measures. All published studies with an adequate sample size reported approval, and an appropriate comparator was more commonly used in these studies than in studies that did not refer to the ethics committee (12/22 v 7/25 studies respectively).
Conclusions
Our proposed protocol showed several important ethical flaws in the studies on which we base our current management for cancer patients with febrile neutropenia. For most measures, the protocol enabled straightforward and systematic extraction of data on ethical issues. The details of how to assess several of the measures, such as the justification of studies or the appropriateness of the comparators, may need further consideration to allow for uniform evaluation in systematic reviews of different sorts.
Summary points
Many clinical trials do not report details of ethical issues
Including ethical assessment in systematic reviews will encourage researchers to conduct ethical research
Ethical research may provide more reliable results
Ethics can be assessed systematically by using the suggested protocol
We believe that including ethical considerations in systematic reviews will increase research workers' awareness of the need to conduct ethical research. It might also contribute to the interpretation of results. To encourage implementation of this proposal, we recommend that the next CONSORT statement should include ethical justification as an integral part of their protocol for reporting trials.15
We thank Iain Chalmers, Andrew Herxheimer, and Amanda Burls for their comments and suggestions.
Contributors and sources: MAW was a founder member of the Cochrane Colaboration primary care field and teaches medical ethics. LL is a specialist in infectious disease and has written systematic reviews. MP wrote the systematic review that is used to illustrate this article
Competing interests: None declared.
References
Bauchner H, Scharshtein J. Failure to report ethical approval in child health research: review of published papers. BMJ 2001;323: 318-9.
Pappworth M. Human guinea pigs: experimentation on man. Boston: Beacon, 1967.
Seidelman WE. Mengele medicus: medicine's Nazi heritage. Milbank Q 1988;66: 221-39.
Weitzman M. The ethics of using Nazi medical data. Second Opinion: Health Faith and Ethics 1990;14: 27-39.
Ruiz-Canela M, Irala-Estevez J, Martinez-Gonzalez MA, Gomez-Gracia E, Fernandez-Crehuet J. Methodological quality and reporting of ethical requirements in clinical trials. J Med Ethics 2001;27: 172-6.
Foster C. The ethics of medical research on humans. Cambridge: Cambridge University Press, 2001.
Freedman B, Fuks A, Weijer C. In loco parentis. Minimal risk as an ethical threshold for research on children. Hastings Cent Rep 1993;23(2): 13-9.
Doyal L, Tobias JS, eds. Informed consent in medical research. London: BMJ Publications, 2000.
Paul M, Soares-Weiser K, Leibovici L. lactam monotherapy versus lactam aminoglycoside combination therapy for febrile neutropenia: systematic review and meta-analysis. BMJ 2003;326: 1111-5.
Jones B, Jarvis P, Lewis JA, Ebbutt AF. Trials to assess equivalence: the importance of rigorous methods. BMJ 1996;313: 36-9.
Wade JC, Bustamante C, Delin A, Finely R, Drusano G, Thompson B. Imipenem vs piperacillin plus amikacin, empiric therapy for febrile neutropenic patients: a double blind trial . Abstracts of the 27th Interscience Conference on Antimicrobial Agents and Chemotherapy. Washington, DC: American Society for Microbiology, 1987.
Hughes WT, Armstrong D, Bodey GP, Feld R, Mandell GL, Meyers JD, et al. From the Infectious Diseases Society of America. Guidelines for the use of antimicrobial agents in neutropenic patients with unexplained fever. J Infect Dis. 1990;161: 381-96.
Hughes WT, Armstrong D, Bodey GP, Brown AE, Edwards JE, Feld R, et al. 1997 guidelines for the use of antimicrobial agents in neutropenic patients with unexplained fever. Clin Infect Dis 1997;25: 551-73.
Hughes WT, Armstrong D, Bodey GP, Bow EJ, Brown AE, Calandra T, et al. 2002 guidelines for the use of antimicrobial agents in neutropenic patients with cancer. Clin Infect Dis 2002;34: 730-51.
CONSORT statement for reporting randomised trials. www.consortstatement.org (accessed 24 Feb 2004).(Michael A Weingarten, pro)
Correspondence to: M A Weingarten weingml@post.tau.ac.il
Routine assessment may improve ethical standards and overall quality of trials
Our awareness of the requirements for ethical clinical research has increased over the past century. Research ethics committees were set up after the Declaration of Helsinki to review research proposals. Many journals now require a statement that ethical approval has been obtained before they consider a research report for publication. Nevertheless, many published studies do not come up to standard, or at least do not report that they do. For example, 30 out of 37 consecutive studies published in five general paediatric journals did not report whether informed consent was obtained. Twenty four of them did not report whether the committee on research ethics had approved the study.1 We propose that systematic reviews of experimental clinical research on humans should also include information on the ethical standards of the trials.
Why include ethical information?
The main reason for including ethics in the checklist of systematic reviews is to increase awareness in the scientific community about the need for high ethical standards in research on humans. The proposal would also encourage reviewers to identify those occasional studies that were so unethical that there may be doubts about the morality of using the results. Although such trials are rare, history has given us too many real examples to allow us to be complacent.2 3 Opinions differ on whether it is justified to disseminate the results of such studies.4 Either way, a conscious decision should be made and revealed to readers of the review.
Issues around ethical quality overlap importantly with the central issues of the validity, reliability, and generalisability of research findings. They relate to some of the more subtle potential sources of bias in experimental clinical research. It is thus important to include ethical assessment in systematic reviews on prudential grounds as well as on moral grounds. The results of ethically sound trials may be more reliable than those from other trials. In a review of 767 randomised controlled trials published during 1993-5 in the New England Journal of Medicine, Lancet, BMJ, and JAMA, trials of higher methodological quality were more likely to provide information about their ethical aspects; so it seems that more reliable research is also more sensitive to the ethical requirements of research.5
Which issues should we assess?
Ethical issues at the level of a systematic review are different from those for original research, and it might be unfair to examine all trials under the same ethical loupe. Older trials will have been done when ethical standards of research had not been as clearly formulated as they are today. Ethical standards depend not only on the time the study was performed but also on place and social mores.
Nevertheless, we think that examination of the ethical standards of trials should form part of every review, but the ethical principles to be applied need to be defined. Among the approaches to research ethics, we find that proposed by Foster most attractive. She suggests examining the project from three perspectives: goals, duties, and rights.6 We used this model to devise a protocol to help assess the ethical quality of clinical trials included in systematic reviews (box).
Should systematic reviews merely delineate the ethical quality or attempt to integrate the observations into its conclusions? We believe that reviews should at least include a report of the ethical assessment and that the implications for further research should address the ethical gaps observed in existing studies.
How would the protocol work in practice?
We applied our protocol to a recently published systematic review of trials comparing lactam monotherapy versus lactam combined with aminoglycoside for febrile neutropenia.9 The review included 47 studies carried out during 1981-2000 and 7807 randomised patients.
Goal related considerations
Financial support—Twenty eight trials gave information on financial support: all had received funding from the company that manufactured the monotherapy antibiotic. Three trials declared research grant support as well as commercial support.
Conflicts of interest—None of the studies included a statement related to potential conflicts of interest.
Justification—Thirty seven studies did not report a sample size calculation. We calculated that an equivalence trial design with an overall success rate of 60% (as observed in this review), and permitting a 20% difference between the treatments, would require around 200 assessable patients to ensure an 80% chance of rejecting at = 0.05 the null hypothesis of equivalence.10 Thirty eight trials had fewer than 200 patients. This requirement is modest given the broad difference permitted between the treatments. Compilation of all existing data in this review led us to believe that further trials comparing lactam monotherapy with a narrower spectrum lactam combined with an aminoglycoside would be unjustified. However, we could not identify a specific time at which this conclusion should have been reached.
Guide for ethical assessment of trials in systematic reviews
Goal related considerations
Is there a clear declaration on financial support in all trials?
Is there a statement that relates to potential conflicts of interest in all trials?
Justification—Could the results have been obtained by laboratory or animal experiments? Were any of the trials superfluous? Was the size of the study sufficient to achieve adequate statistical power?
Publication bias—How many of the identified trials remained unpublished? Is bias detectable by funnel plot analysis?
Duty related considerations
? Were the comparators appropriate? If a placebo was used, was it justified?
Rights related considerations
Safety—Was the risk for participants appropriate to the importance of the research?7 Was appropriate follow up care assured?
Was informed consent obtained?
When participants had reduced competence, were appropriate measures taken to protect their best interests?8
Were adequate steps taken to prevent unauthorised access to personal and clinical data?
Global considerations
? Was the study approved by a research ethics committee?
Publication bias—Five trials completed before 1990 (943 patients) remained unpublished or were published only as conference proceedings. The results of one study including 460 patients were never formally published.11 Although we included these studies in our review, the funnel plot showed that small studies favouring combination therapy might be missing.
Duty related considerations
Appropriate comparators—We considered antibiotic combinations advised by guidelines contemporary to the study years as adequate.12-14 Twenty trials used an inadequate comparator on this basis.
Rights related considerations
Safety—We considered monitoring aminoglycoside serum concentrations at any time during the study and creatinine at least twice weekly as minimal follow up measures to ensure patients' safety. Thirty one trials did not report aminoglycoside monitoring and 17 had inadequate creatinine follow up.
Informed consent—Twenty nine trials reported that informed consent had been obtained from adult patients.
Participants of reduced competence—Twenty five trials included patients younger than 16 years old, and in only five of these was consent obtained from parents or relatives. The trials covered a variety of patient populations, including some seriously ill patients with disseminated carcinoma and geriatric patients. Consent was sometimes given by the family rather than the patient, and in those cases it was not clear whether the family had any power of attorney to act on the patient's behalf.
Confidentiality—There was no information on steps taken to ensure confidentiality in the published reports.
Global considerations
Approval by research ethics committee—Twenty two studies stated that they had been approved by a research ethics committee. We noted an association between this approval and other ethical measures. All published studies with an adequate sample size reported approval, and an appropriate comparator was more commonly used in these studies than in studies that did not refer to the ethics committee (12/22 v 7/25 studies respectively).
Conclusions
Our proposed protocol showed several important ethical flaws in the studies on which we base our current management for cancer patients with febrile neutropenia. For most measures, the protocol enabled straightforward and systematic extraction of data on ethical issues. The details of how to assess several of the measures, such as the justification of studies or the appropriateness of the comparators, may need further consideration to allow for uniform evaluation in systematic reviews of different sorts.
Summary points
Many clinical trials do not report details of ethical issues
Including ethical assessment in systematic reviews will encourage researchers to conduct ethical research
Ethical research may provide more reliable results
Ethics can be assessed systematically by using the suggested protocol
We believe that including ethical considerations in systematic reviews will increase research workers' awareness of the need to conduct ethical research. It might also contribute to the interpretation of results. To encourage implementation of this proposal, we recommend that the next CONSORT statement should include ethical justification as an integral part of their protocol for reporting trials.15
We thank Iain Chalmers, Andrew Herxheimer, and Amanda Burls for their comments and suggestions.
Contributors and sources: MAW was a founder member of the Cochrane Colaboration primary care field and teaches medical ethics. LL is a specialist in infectious disease and has written systematic reviews. MP wrote the systematic review that is used to illustrate this article
Competing interests: None declared.
References
Bauchner H, Scharshtein J. Failure to report ethical approval in child health research: review of published papers. BMJ 2001;323: 318-9.
Pappworth M. Human guinea pigs: experimentation on man. Boston: Beacon, 1967.
Seidelman WE. Mengele medicus: medicine's Nazi heritage. Milbank Q 1988;66: 221-39.
Weitzman M. The ethics of using Nazi medical data. Second Opinion: Health Faith and Ethics 1990;14: 27-39.
Ruiz-Canela M, Irala-Estevez J, Martinez-Gonzalez MA, Gomez-Gracia E, Fernandez-Crehuet J. Methodological quality and reporting of ethical requirements in clinical trials. J Med Ethics 2001;27: 172-6.
Foster C. The ethics of medical research on humans. Cambridge: Cambridge University Press, 2001.
Freedman B, Fuks A, Weijer C. In loco parentis. Minimal risk as an ethical threshold for research on children. Hastings Cent Rep 1993;23(2): 13-9.
Doyal L, Tobias JS, eds. Informed consent in medical research. London: BMJ Publications, 2000.
Paul M, Soares-Weiser K, Leibovici L. lactam monotherapy versus lactam aminoglycoside combination therapy for febrile neutropenia: systematic review and meta-analysis. BMJ 2003;326: 1111-5.
Jones B, Jarvis P, Lewis JA, Ebbutt AF. Trials to assess equivalence: the importance of rigorous methods. BMJ 1996;313: 36-9.
Wade JC, Bustamante C, Delin A, Finely R, Drusano G, Thompson B. Imipenem vs piperacillin plus amikacin, empiric therapy for febrile neutropenic patients: a double blind trial . Abstracts of the 27th Interscience Conference on Antimicrobial Agents and Chemotherapy. Washington, DC: American Society for Microbiology, 1987.
Hughes WT, Armstrong D, Bodey GP, Feld R, Mandell GL, Meyers JD, et al. From the Infectious Diseases Society of America. Guidelines for the use of antimicrobial agents in neutropenic patients with unexplained fever. J Infect Dis. 1990;161: 381-96.
Hughes WT, Armstrong D, Bodey GP, Brown AE, Edwards JE, Feld R, et al. 1997 guidelines for the use of antimicrobial agents in neutropenic patients with unexplained fever. Clin Infect Dis 1997;25: 551-73.
Hughes WT, Armstrong D, Bodey GP, Bow EJ, Brown AE, Calandra T, et al. 2002 guidelines for the use of antimicrobial agents in neutropenic patients with cancer. Clin Infect Dis 2002;34: 730-51.
CONSORT statement for reporting randomised trials. www.consortstatement.org (accessed 24 Feb 2004).(Michael A Weingarten, pro)