Research-Design Issues in Cancer-Symptom–Management Trials Using Complementary and Alternative Medicine: Lessons From the National Cancer In
http://www.100md.com
《临床肿瘤学》
the National Cancer Institute, Bethesda, MD
ABSTRACT
PURPOSE: To identify major research-design issues in proposals submitted by investigators in the Community Clinical Oncology Program (CCOP) for clinical trials of complementary and alternative medicine (CAM) for cancer-symptom management.
METHODS: We conducted content analysis of all scientific reviews of concepts and protocols submitted by the CCOP to the National Cancer Institute (NCI) to identify research challenges in conducting clinical trials designed to evaluate CAM interventions for cancer-symptom management.
RESULTS: Since the inception of the NCI Office of Cancer Complementary and Alternative Medicine in 1998, a total of 46 symptom-management studies using CAM interventions have been proposed by CCOP investigators, with 20 studies now in progress comprising 22% of the current total CCOP symptom-management portfolio. Proposals fell into four categories: complex natural products; nutritional therapeutics; mind-body interventions; and alternative medical systems. The most significant research-design issues arose as a consequence of the lack of preclinical data for CAM interventions and the lack of quality-control standards comparable with those used in regulating new pharmaceutical agents.
CONCLUSION: Across the different types of CAM interventions, the most common problems found in proposed research designs are related to unwarranted assumptions about the consistency and standardization of CAM interventions, the need for data-based justifications for the study hypotheses, and the need to implement appropriate quality control and monitoring procedures during the course of the trial. To advance the state of the science, future research must address these critical issues if CAM interventions are to be evaluated rigorously and have a consequent impact on clinical practice and general public awareness.
INTRODUCTION
Many cancer patients perceive a benefit to the use of complementary and alternative medicine (CAM), but many CAM products and procedures have not been tested adequately to determine their safety and efficacy. Even less is known about the interaction of CAM interventions with chemotherapy and radiation. With technical support, scientific review, and funding from the National Cancer Institute (NCI) Office of Cancer Complementary and Alternative Medicine (OCCAM), the NCI-supported Community Clinical Oncology Program (CCOP) has been conducting clinical trials to evaluate CAM interventions for cancer-symptom management since 1998. This article reports on major challenges in designing clinical trials to answer questions about the safety and efficacy of CAM interventions in cancer-symptom–management research.
Estimates of the percentage of the US adult population who have used CAM vary widely (from 30% to 80%) according to different reports and different definitions of CAM.1-3 In the most recent nationally representative survey of 31,044 adults conducted by the Centers for Disease Control, 36% reported current use of CAM, a rate that rises to 62% when megavitamins and prayer for health reasons are included.4 CAM use was found to be more common among white, well-educated females than other population groups. In 1997, Americans spent an estimated $36 to $47 billion on CAM therapies.3
Rates of CAM use are exceptionally high among patients with cancer or other life-threatening illnesses.5 Several reports have noted that cancer patients underreport CAM use in medical histories and that oncologists underestimate the rates of CAM use in their patients.6-8 Surveys have found that 70% to 85% of cancer patients have used at least one CAM approach in conjunction with their cancer treatment.9,10 Other reports have indicated that cancer patients have taken the lead in using CAM for symptom management, including pain, anxiety, stress management, nausea and vomiting, and other symptoms.11 The use of CAM therapies for symptom management seems to be higher among patients with greater psychosocial distress and poorer quality of life.12,13
A single, commonly accepted definition of CAM is difficult to provide. In the Common Scientific Outline developed by the NCI Office of Science Planning and Assessment for the purposes of analyzing research resource allocations, CAM is defined as "forms of treatment that are used in addition to (complementary) or instead of (alternative) standard treatments. These practices generally are not considered standard medical approaches. CAM may include dietary supplements, megadose vitamins, herbal preparations, special teas, acupuncture, massage therapy, magnet therapy, spiritual healing, and meditation."14 There are several systems for categorizing CAM interventions. Based on the classification system used at OCCAM, CAM therapies are divided into six categories: (1) alternative medical systems; (2) energy therapies; (3) manipulative and body-based methods; (4) mind-body interventions; (5) nutritional therapeutics; and (6) pharmacologic and biologic treatments (including complex natural products15; Table 1 16).
In 1998, the NCI established the OCCAM to advance the scientifically rigorous study of CAM modalities for the diagnosis, prevention, and treatment of cancer and its symptoms. A major goal of OCCAM is to integrate rigorous CAM research into all divisions of the NCI.17,18 OCCAM promotes CAM research in the CCOP by issuing an annual solicitation for proposals for supplemental funding to the CCOP research bases and by providing scientific expertise in the review of CAM concepts and protocols submitted by the CCOP research bases.
The CCOP is a clinical trials network supported by the NCI, with funding for cooperative groups and cancer centers selected as research bases to design and conduct clinical trials in cancer control and prevention. Currently, eight cooperative groups and six cancer centers are funded as CCOP research bases (Table 2). The CCOP also funds community physicians to accrue patients to these trials.
All trials conducted in the CCOP network are reviewed and approved by the NCI before implementation. Similar to other symptom-management trials conducted by the CCOP, investigators at the CCOP research bases develop protocols for CAM studies based on their clinical interests and prior research experience. Although the NCI can suggest areas for study, NCI involvement focuses primarily on the review of submitted concepts and protocols. The majority of these trials are phase III clinical trials. The NCI uses a two-stage process in the review of CCOP proposals. First, a concept-review committee reviews a study concept, reporting its results to the research base in a review letter. Reviewers can reject the concept, request modifications, or approve it for protocol development. If approved, the research base submits a protocol for additional review. At this level, reviewers also have the option of rejecting the proposal, requesting modifications, or granting final approval. Both levels of review generate one or more letters summarizing the reviewers' concerns and recommendations for any given study. These scientific review letters constitute the primary data analyzed for this investigation.
The purpose of this investigation was to identify major research challenges in designing phase III trials of CAM agents for treatment of cancer symptoms. We do not report on trial results, because most are still open and results have not been published. This study does not address the use of CAM therapies in cancer treatment or prevention trials or investigator-initiated symptom-management studies supported through research project grants. This article also does not delve into any philosophical challenges to the validity of the scientific method in assessing CAM interventions as some CAM advocates have done.19-21
METHODS
The data for this report were abstracted from the scientific reviews of all concepts and protocols to evaluate CAM symptom-management interventions submitted by the CCOP research bases since the inception of the OCCAM, from October 1998 through December 2003. To identify CAM concepts and protocols, we used the definition and criteria found in the common scientific outline. Collectively, the authors reviewed all CAM symptom-management research proposals submitted to the CCOP during the period under investigation.
The unit of analysis for this article is the clinical trial and consists of all review documents completed for a particular proposal. It is important to note that these letters include both the reviewers' concerns and often their recommendations for addressing these concerns. From these review letters, the major concerns and recommendations regarding research design identified by the reviewers were categorized by using standard methods of content analysis.22 All scientific reviews were analyzed initially by D.R.B., and a code book of major themes was developed. The entire research team then coded separate subsets of the reviews independently. After review and revision of the coding scheme, the entire sample was then recoded and verified by the entire research team. These data were not quantified, because the purpose of the research was to identify the full range of issues that investigators must address to conduct valid and reliable CAM clinical trials.
RESULTS
We identified a total of 46 CAM proposals submitted by the CCOP research bases during the period October 1998 through December 2003. Studies were categorized as complex natural products (n = 16), nutritional therapeutics (n = 15), mind-body interventions (n = 12), and alternative medical systems (n = 3; Table 3). No proposals were submitted to test energy therapies (eg, therapeutic touch, Qigong) or manipulative and body-based methods (eg, massage techniques). Of the total 46 proposed CAM studies, 20 trials (22% of the current CCOP symptom-management portfolio) were active at the time of this research (2004). The remaining 26 were closed, under revision, or not approved.
In our analysis of the review letters, the most challenging research-design issues stemmed from the fact that CAM interventions are available over the counter without regulatory oversight or the same quality controls that govern the distribution of pharmaceutical agents. Literature reviews frequently were faulted for failing to ascertain the precise CAM process or product reported in previous studies; these therapies cannot be assumed to be equivalent, because CAM therapies are largely unregulated. Study rationales that establish preliminary efficacy, proper dosage, and safety were generally inadequate, because the preclinical data that typically accompany the development of new pharmaceutical agents were unavailable. Proposals often lacked an explicit quality-control plan from an identified manufacturer or trainer (for mind-body interventions), which is necessary to ensure consistency and comparability. In addition, studies lacked procedures to control for potential drop-ins and sample contamination, which is a particular concern because CAM therapies are readily available in the marketplace. Finally, many investigators did not provide an adequate justification beyond claiming that patients were already using a CAM therapy for some particular purpose. Investigators did not always specify a plausible hypothesis based on previous research or an explanation of the mechanism of action.
Five major concerns regarding the characteristics of the intervention itself were identified: (1) study justification; (2) standardization and quality control of the intervention; (3) appropriate dose; (4) adverse effects and interactions with other drugs; and (5) the burden of the intervention on patients and health care providers (Table 4). In addition, we identified a number of measurement and design issues. These issues are common to all types of research investigations, but presented unusual challenges caused by the unique characteristics of CAM interventions. Measurement and design issues included (1) identifying an appropriate comparison group, (2) identifying appropriate end points, (3) justifying the anticipated effect sizes used to determine sample size, (4) identifying an appropriate timeline for assessments, (5) subject and clinician blinding to the treatment arm, and (6) sample contamination caused by drop-ins. We describe the intervention issues first, organized by CAM modality, followed by a discussion of measurement and design issues.
Nutritional Therapeutics and Complex Natural Products
Because the implications for study design are similar, we combined nutritional therapeutics and complex natural products as ingestible substances taken for therapeutic intent. Together, they were the most common CAM intervention modality in the CCOP proposals, accounting for more than two thirds of the CAM protocols.
Starting with study justification, many proposals failed to provide sufficient data to justify the conduct of a phase III trial. For example, reviewers questioned whether there were sufficient preliminary data to indicate that gingko biloba would improve cognitive functioning, that ginger would reduce nausea and vomiting, or that St John's wort would reduce hot flashes. Justification of the proposed dose was also deficient in many proposals in this category.
In the review letters, the most significant concern with this CAM modality was that, unlike pharmaceutical agents, ingestible CAM products are not manufactured in ways that guarantee that they are unadulterated and identical to one another. Lack of regulation made the interpretation of previous studies difficult and presented atypical concerns regarding product consistency (eg, concentration of active ingredients) and contamination. The lack of standardization belied the assumption that products evaluated in one trial were the same as those tested in another or that the results using one preparation apply to another preparation. For example, because several different ginger-extract products have been studied, researchers were encouraged to use that product for which there was the most information to build on the results of previous research and strengthen the study justification.
Given consistent concerns about standardization, researchers were uniformly asked to specify one formulation of a product (eg, powder v pill), one species of herb versus another, and one source for obtaining the product. Before approval, all proposals to evaluate these products were required to identify the manufacturer, who had to provide assurances of standardized doses, information about stability of the product, and any special storage conditions that would be needed during the trial. If possible, researchers were encouraged to use a single lot of the agent throughout the trial. Researchers were also asked to consider independent analysis of the product for contamination, especially for pesticides or heavy metals. (One case in point from a non-CCOP study, the herbal dietary supplement PC SPES was found to contain the undeclared prescription drug ingredients warfarin and DES23; thus, these products may have effects unrelated to the CAM agent under investigation.) Researchers were also advised to consider conducting studies of biologic products under an investigator-initiated investigational new drug application with the US Food and Drug Administration.
Many protocols were criticized for lack of data on potential adverse effects of nutritional products such as carnitine, creatine, or coenzyme Q10 and, more specifically, the potential interactions between these agents and chemotherapy. Researchers were asked to provide as much information as possible about anticipated toxicities and define grades of toxicities that would trigger discontinuation. Concerns were raised that many nutritional therapeutic interventions may be contraindicated for patients with pre-existing medical conditions (eg, bleeding disorders or patients taking warfarin or heparin24). Researchers were cautioned that certain agents may even protect tumors by interfering with the metabolism of chemotherapeutic agents.25 St John's wort, for example, has been found to interfere with irinotecan metabolism.26
Mind-Body Interventions
Mind-body interventions (n = 12; Table 3) ranged from relaxation techniques and stress management to various exercise programs, expressive writing, and support-group therapy. Although investigators often cited a fairly large literature on this intervention modality to support the study justification, reviewers raised concerns about their interpretation of previous studies. They cautioned that various stress-management, exercise, and writing interventions reported in the literature could not be considered equivalent and interchangeable. Reviewers of mind-body proposals cited major concerns with standardization, justification of the identified dose, and schedule and length of treatments.
One particular concern of the reviewers was that such "operator-dependent" therapies lacked sufficient discussion of procedures to be used to ensure standardization and quality control, a problem that perhaps was even greater in this domain than in nutritional products. Proposals needed to specify the training and credentials of the mind-body therapist. Research protocols frequently failed to demonstrate adequate training procedures to ensure uniform interventions throughout the trial, discuss requirements for ongoing training, and outline how these processes would be monitored to assure adherence. Reviewers were concerned, for instance, that some nurses would be more skilled in delivering mindfulness relaxation techniques than others; proposed study designs often did not discuss how they would address this potential variability. Similarly, researchers were asked to demonstrate how behavioral interventions would be standardized across different providers and intervention sites. A related problem in mind-body interventions was an inadequate description of procedures to monitor different doses, for example, length of time spent writing each week or intensity and duration of exercise.
Reviewers were also concerned about the potential for adverse events with many mind-body interventions, particularly exercise programs or guided imagery (eg, dissociative experiences).27 Researchers were asked to provide information on the suitability and safety of the interventions for the study population (eg, potential interactions between exercise interventions and treatment-related chemotherapeutic toxicities including fatigue, nausea, vomiting, and diarrhea). Researchers also were requested to submit plans for modifying an exercise program to adjust for the physical abilities and limitations of patients and to provide plans for the analysis that take into account potential variations in dosage.
One particular concern with mind-body interventions was that many are highly labor intensive. The problem was that the time burden might interfere with the willingness of staff and patients to participate, thus limiting general clinical applicability. Many mindfulness relaxation techniques also required extensive staff training. Hence, proposals were directed to demonstrate that sufficient numbers of staff with adequate training were available to carry out the study and that patients would not be overly burdened. Proposals also needed to address whether the nurse (or other practitioner) and patient would have sufficient time to develop an effective relationship in conducting mindfulness relaxation exercises and other similar procedures.
Alternative Medical Systems
In the category of alternative medical systems, the CCOP investigators submitted proposals to study acupressure, acustimulation, and a homeopathic intervention, Traumeel S. Because acupressure and acustimulation are taught to patients by trained staff and administered by the patients themselves, standardization and quality control were major concerns identified by reviewers. Concerns about standardization focused on the need to specify a precise and uniform amount of the intervention: the frequency at which the acupressure device would be set; number of times per day and per week that the treatment would be administered; precise site of stimulation; and support for these decisions with data. In terms of quality control, proposals were required to demonstrate that the training plan for staff and patients was adequate to prepare them to administer the intervention reliably over time. Regarding potential adverse effects, reviewers' concerns centered on the need to specify procedures for adjusting the dose if subjects experienced toxicity (eg, nausea and vomiting).
For homeopathic interventions, the most significant issue was study justification: providing empirical data to explain why the proposed intervention could be expected to be effective. For quality control purposes, researchers were asked to address questions about shelf life and storage conditions to ensure consistency of the intervention over time and to provide a plan to assess patient compliance with the treatment regimen. The practice of homeopathy is based on its proposed "law of similars," which suggests that "like cures like." That is, a substance that causes specific symptoms in a healthy person could cure those same symptoms in a sick person. Homeopathic medicines are intended to enable the body to initiate the healing process rather than eliminate the symptoms. Homeopathic medicines are traditionally very small doses of natural substances that have been diluted in water and/or alcohol many times.
Measurement and Design Issues
Most measurement and design issues identified in our analysis are common to all clinical trials of symptom-management interventions but raise particular challenges because of the unusual characteristics of CAM interventions. A recurring concern was identifying an appropriate control intervention. For example, to rule out potential placebo effects, interactive interventions (eg, relaxation techniques, acupressure, etc) needed to specify a control intervention that differed from the study intervention only in the essential elements addressed in study hypotheses but not in other aspects such as attention from study personnel or time spent on the procedure. Examples were the use of a sham acupressure technique28 to allow the evaluation of the correct placement or the use of relaxing music as a comparison condition for mindfulness relaxation. Evaluation of complex natural products that have distinctive odors (such as ginger) presented yet another challenge in identifying an appropriate comparison group. On the one hand, reviewers were concerned that a control intervention that had no odor might elicit suspicion from control participants. Conversely, a control intervention that had a distinctive odor might exaggerate any potential placebo effects. Investigators were encouraged to use products that have no odor when available.
Mind-body interventions also presented particular challenges in determining appropriate end points. An example was whether a mindfulness relaxation technique directly affects the symptoms (eg, number of incidents of vomiting) or merely affects the subject's perceptions of symptoms (eg, how bothersome they are). Other questions included whether a particular product (such as coenzyme Q10) was intended to prevent or relieve fatigue or whether a product (such as valerian) was hypothesized to impact both sleep quality and fatigue or one but not the other.
In the absence of substantial preliminary data regarding the onset and duration of the effects (eg, of acupressure or homeopathic preparations), study designs often failed to specify and justify an optimal time for conducting follow-up assessment. Similarly, in conducting power analyses to compute the sample size, the stated effect size often needed additional justification.
In proposals for which the same staff are responsible for conducting comparison interventions, researchers had to address how they would prevent contamination to ensure, for example, that any mindfulness relaxation techniques did not inadvertently spill over into interactions with patients in the comparison group.
Finally, because many CAM products and procedures are available outside the medical setting, another major concern was the problem of "drop-ins" and sample contamination. Investigators were asked to develop a careful plan to monitor whether patients might seek out CAM products or practices on their own.
DISCUSSION
Explanations for the growth in CAM use include marketing forces, the desire of patients to be actively involved in medical decision making, and dissatisfaction with conventional Western medicine.4 Most CAM therapies, however, have not been tested rigorously to determine their efficacy in reducing symptoms. Worse, little is known about their safety in combination with conventional cancer therapies. Despite wide public acceptance, it is critically important to review the scientific literature on these products, carefully assess risk-benefit ratios, and design research investigations appropriate to current levels of knowledge.
One finding of this research, the predominance of interventions using ingestible substances over other CAM modalities, deserves comment. Because the list of CAM interventions listed in Table 3 consists of all CAM studies submitted by the CCOP to the NCI and does not reflect the effects of the NCI review, the distribution of studies reflects the investigators' interests and their internal review processes. Although we have no data to support the assumption, CCOP investigators are clinicians trained in Western medicine and may be more comfortable with CAM interventions that resemble medical practices used in their training and experience. This conjecture points to a limitation of the research. Because the study sample was limited to investigations initiated by the CCOP, it does not represent the universe of CAM research, as reflected in the skewed distribution of studies across the different CAM modalities. To increase the range of CAM studies (eg, in energy therapies or manipulative methods), investigators may need to form partnerships with experienced CAM practitioners to explore their potential for cancer-symptom relief. Such partnerships would undoubtedly raise new research-design issues. It is worth noting, for example, that this sample of studies did not propose the use of any CAM inclusion or outcome criteria such as the patient's vital force, pulse or tongue diagnoses, or suggestibility in mind-body interventions, to name but a few. The operationalization and measurement of these concepts would present new challenges but might open the field to new discoveries as well.
A number of research-design issues that need to be addressed to advance the state of the science on CAM interventions have been identified with the results of this research. The most significant concerns raised by the reviewers focused on the need to (1) justify the proposed intervention, specifying plausible data-based hypotheses regarding the clinical effects of the proposed intervention, (2) standardize the intervention by gaining agreements with the manufacturers beforehand and implementing appropriate quality-control procedures during the course of the trial, and (3) monitor the comparison group to avoid sample contamination. Although the results of similar studies (eg, effect of a CAM product on fatigue in patients with heart disease) provide grounds for formulating a plausible hypothesis, proposals could be strengthened greatly by specifying the active ingredient and hypothesized mechanism of action. Because the usual preclinical data found in research on new pharmaceutical agents are generally not available for many CAM interventions, researchers may want to consider conducting pilot studies before moving into phase III efficacy trials.
Because "natural" products may have many active ingredients, with varying concentrations between products and even between different batches of the same product, they cannot be assumed to be safe simply because they are already in widespread use. The physiologic and pharmacologic implications of administering these products to ill patients who are also receiving other medications must always be considered carefully. As with other new pharmaceutical agents, researchers should contact the US Food and Drug Administration regarding the need for an investigational new drug application for any ingestible CAM product.
This report highlights many of the complexities of conducting rigorous scientific research on CAM products and practices. Although the issues raised illustrate how research on CAM interventions may be more difficult than studies of new pharmaceutical agents, it is important for future research protocols to address these complexities directly. Stronger research designs are essential to evaluate CAM interventions rigorously and influence clinical practice and general public awareness.
Authors' Disclosures of Potential Conflicts of Interest
The authors indicated no potential conflicts of interest.
NOTES
Authors' disclosures of potential conflicts of interest are found at the end of this article.
REFERENCES
Ni H, Simile C, Hardy AM: Utilization of complementary and alternative medicine by United States adults: Results from the 1999 National Health Interview Survey. Med Care 40:353-358, 2002
Wetzel M, Kaptchuk T, Haramati A, et al: Complementary and alternative medical therapies: Implications for medical education. Ann Intern Med 138:191-197, 2003
Eisenberg D, David R, Ettner S, et al: Trends in alternative medicine use in the United States, 1990-1997. JAMA 289:1569-1575, 1998
Barnes PM, Powell-Griner E, McFann K, et al: Complementary and alternative medicine use among adults: United States 2002. Adv Data 343:1-20, 2004
Sparber A, Wootton JC: Surveys of complementary and alternative medicine: Part II. Use of alternative and complementary cancer therapies. J Altern Complement Med 7:281-287, 2001
Metz J, Jones H, Devine P, et al: Cancer patients use unconventional medical therapies far more frequently than standard history and physical examination suggest. Cancer J 7:149-154, 2001
Jones H, Metz J, Devine P, et al: Rates of unconventional medical therapy use in patients with prostate cancer: Standard history versus directed questions. Urology 59:272-276, 2002
Kao D, Devine P: Use of complementary health practices by prostate carcinoma patients undergoing radiation therapy. Cancer 88:615-619, 2000
Miller M, Boyer M, Butow P, et al: The use of unproven methods of treatment by cancer patients: Frequency, expectations and cost. Support Care Cancer 6:337-347, 1998
Richardson M, Sanders T, Palmer J, et al: Complementary/alternative medicine use in a comprehensive cancer center and the implications for oncology. J Clin Oncol 18:2505-2521, 2000
Ott MJ: Complementary and alternative therapies in cancer symptom management. Cancer Pract 10:162-166, 2002
Paltiel O, Avitzour M, Peretz T, et al: Determinants of the use of complementary therapies by patients with cancer. J Clin Oncol 19:2439-2448, 2001
Burstein H, Gelber S, Guadagnoli E, et al: Use of alternative medicine by women with early-stage breast cancer. N Engl J Med 340:1733-1739, 1999
National Cancer Institute: Cancer research portfolio. Available at: http://researchportfolio.cancer.gov.
White JD: Complementary, alternative, and unproven methods of cancer treatment, in DeVita VT, Hellman S, Rosenberg SA (eds): Cancer: Principles and Practice of Oncology (6th ed). Philadelphia, PA, Lippincott, Williams & Wilkins, 2001, pp 3147-3157
National Cancer Institute, Office of Cancer Complementary and Alternative Medicine: Understanding CAM. Available at: http://cancer.gov/cam/health_understanding.html
Office of Cancer Complementary and Alternative Medicine: Expert Panels in Cancer CAM Research: Developing the State of the Science in Research Methodologies. Bethesda, MD, Department of Health and Human Services, National Institutes of Health, National Cancer Institute, DHHS publication 03-5427, 2003
Smith W: Research methodology: Implications for CAM pain research. Clin J Pain 20:3-7, 2004
Schaffner KF: Assessments of efficacy in biomedicine: The turn toward methodological pluralism, in Callahan D (ed): The Role of Complementary and Alternative Medicine: Accommodating Pluralism. Washington, DC, Georgetown University Press, 2002
Hufford D: CAM and cultural diversity: Ethics and epistemology converge, in Callahan D (ed): The Role of Complementary and Alternative Medicine: Accommodating Pluralism. Washington, DC, Georgetown University Press, 2002
O'Connor B: Personal experience, popular epistemology, and complementary and alternative medicine research, in Callahan D (ed): The Role of Complementary and Alternative Medicine: Accommodating Pluralism. Washington, DC, Georgetown University Press, 2002
Miles M, Huberman A: Qualitative Data Analysis: An Expanded Sourcebook (2nd ed). Thousand Oaks, CA, Sage Publications, 1994
White J: PC-SPES: A lesson for future dietary supplement research. J Natl Cancer Inst 94:1261-1262, 2002
Dasgupta A: Review of abnormal laboratory test results and toxic effects due to use of herbal medicines. Am J Clin Pathol 120:127-137, 2003
Sparreboom A, Cox MC, Acharya MR, et al: Herbal remedies in the United States: Potential adverse interactions with anticancer agents. J Clin Oncol 22:2489-2503, 2004
Mathijssen RH, Verweij J, de Bruijn P, et al: Effects of St. John's wort on irinotecan metabolism. J Natl Cancer Inst 94:1247-1249, 2002
Persinger MA: Enhanced incidence of "the sensed presence" in people who have learned to meditate: Support for the right hemispheric intrusion hypothesis. Percept Mot Skills 75:1308-1310, 1992 (3 pt 2)
Kaptchuk TJ, Berman B, Mausell RB: The placebo and complementary and alternative medicine: Historical, scientific, and methodological considerations, in Office of Cancer Complementary and Alternative Medicine (ed): Expert Panels in Cancer CAM Research: Developing the State of the Science in Research Methodologies. Bethesda, MD, Department of Health and Human Services, National Institutes of Health, National Cancer Institute, DHHS publication 03-5427, 2003(David R. Buchanan, Jeffre)
ABSTRACT
PURPOSE: To identify major research-design issues in proposals submitted by investigators in the Community Clinical Oncology Program (CCOP) for clinical trials of complementary and alternative medicine (CAM) for cancer-symptom management.
METHODS: We conducted content analysis of all scientific reviews of concepts and protocols submitted by the CCOP to the National Cancer Institute (NCI) to identify research challenges in conducting clinical trials designed to evaluate CAM interventions for cancer-symptom management.
RESULTS: Since the inception of the NCI Office of Cancer Complementary and Alternative Medicine in 1998, a total of 46 symptom-management studies using CAM interventions have been proposed by CCOP investigators, with 20 studies now in progress comprising 22% of the current total CCOP symptom-management portfolio. Proposals fell into four categories: complex natural products; nutritional therapeutics; mind-body interventions; and alternative medical systems. The most significant research-design issues arose as a consequence of the lack of preclinical data for CAM interventions and the lack of quality-control standards comparable with those used in regulating new pharmaceutical agents.
CONCLUSION: Across the different types of CAM interventions, the most common problems found in proposed research designs are related to unwarranted assumptions about the consistency and standardization of CAM interventions, the need for data-based justifications for the study hypotheses, and the need to implement appropriate quality control and monitoring procedures during the course of the trial. To advance the state of the science, future research must address these critical issues if CAM interventions are to be evaluated rigorously and have a consequent impact on clinical practice and general public awareness.
INTRODUCTION
Many cancer patients perceive a benefit to the use of complementary and alternative medicine (CAM), but many CAM products and procedures have not been tested adequately to determine their safety and efficacy. Even less is known about the interaction of CAM interventions with chemotherapy and radiation. With technical support, scientific review, and funding from the National Cancer Institute (NCI) Office of Cancer Complementary and Alternative Medicine (OCCAM), the NCI-supported Community Clinical Oncology Program (CCOP) has been conducting clinical trials to evaluate CAM interventions for cancer-symptom management since 1998. This article reports on major challenges in designing clinical trials to answer questions about the safety and efficacy of CAM interventions in cancer-symptom–management research.
Estimates of the percentage of the US adult population who have used CAM vary widely (from 30% to 80%) according to different reports and different definitions of CAM.1-3 In the most recent nationally representative survey of 31,044 adults conducted by the Centers for Disease Control, 36% reported current use of CAM, a rate that rises to 62% when megavitamins and prayer for health reasons are included.4 CAM use was found to be more common among white, well-educated females than other population groups. In 1997, Americans spent an estimated $36 to $47 billion on CAM therapies.3
Rates of CAM use are exceptionally high among patients with cancer or other life-threatening illnesses.5 Several reports have noted that cancer patients underreport CAM use in medical histories and that oncologists underestimate the rates of CAM use in their patients.6-8 Surveys have found that 70% to 85% of cancer patients have used at least one CAM approach in conjunction with their cancer treatment.9,10 Other reports have indicated that cancer patients have taken the lead in using CAM for symptom management, including pain, anxiety, stress management, nausea and vomiting, and other symptoms.11 The use of CAM therapies for symptom management seems to be higher among patients with greater psychosocial distress and poorer quality of life.12,13
A single, commonly accepted definition of CAM is difficult to provide. In the Common Scientific Outline developed by the NCI Office of Science Planning and Assessment for the purposes of analyzing research resource allocations, CAM is defined as "forms of treatment that are used in addition to (complementary) or instead of (alternative) standard treatments. These practices generally are not considered standard medical approaches. CAM may include dietary supplements, megadose vitamins, herbal preparations, special teas, acupuncture, massage therapy, magnet therapy, spiritual healing, and meditation."14 There are several systems for categorizing CAM interventions. Based on the classification system used at OCCAM, CAM therapies are divided into six categories: (1) alternative medical systems; (2) energy therapies; (3) manipulative and body-based methods; (4) mind-body interventions; (5) nutritional therapeutics; and (6) pharmacologic and biologic treatments (including complex natural products15; Table 1 16).
In 1998, the NCI established the OCCAM to advance the scientifically rigorous study of CAM modalities for the diagnosis, prevention, and treatment of cancer and its symptoms. A major goal of OCCAM is to integrate rigorous CAM research into all divisions of the NCI.17,18 OCCAM promotes CAM research in the CCOP by issuing an annual solicitation for proposals for supplemental funding to the CCOP research bases and by providing scientific expertise in the review of CAM concepts and protocols submitted by the CCOP research bases.
The CCOP is a clinical trials network supported by the NCI, with funding for cooperative groups and cancer centers selected as research bases to design and conduct clinical trials in cancer control and prevention. Currently, eight cooperative groups and six cancer centers are funded as CCOP research bases (Table 2). The CCOP also funds community physicians to accrue patients to these trials.
All trials conducted in the CCOP network are reviewed and approved by the NCI before implementation. Similar to other symptom-management trials conducted by the CCOP, investigators at the CCOP research bases develop protocols for CAM studies based on their clinical interests and prior research experience. Although the NCI can suggest areas for study, NCI involvement focuses primarily on the review of submitted concepts and protocols. The majority of these trials are phase III clinical trials. The NCI uses a two-stage process in the review of CCOP proposals. First, a concept-review committee reviews a study concept, reporting its results to the research base in a review letter. Reviewers can reject the concept, request modifications, or approve it for protocol development. If approved, the research base submits a protocol for additional review. At this level, reviewers also have the option of rejecting the proposal, requesting modifications, or granting final approval. Both levels of review generate one or more letters summarizing the reviewers' concerns and recommendations for any given study. These scientific review letters constitute the primary data analyzed for this investigation.
The purpose of this investigation was to identify major research challenges in designing phase III trials of CAM agents for treatment of cancer symptoms. We do not report on trial results, because most are still open and results have not been published. This study does not address the use of CAM therapies in cancer treatment or prevention trials or investigator-initiated symptom-management studies supported through research project grants. This article also does not delve into any philosophical challenges to the validity of the scientific method in assessing CAM interventions as some CAM advocates have done.19-21
METHODS
The data for this report were abstracted from the scientific reviews of all concepts and protocols to evaluate CAM symptom-management interventions submitted by the CCOP research bases since the inception of the OCCAM, from October 1998 through December 2003. To identify CAM concepts and protocols, we used the definition and criteria found in the common scientific outline. Collectively, the authors reviewed all CAM symptom-management research proposals submitted to the CCOP during the period under investigation.
The unit of analysis for this article is the clinical trial and consists of all review documents completed for a particular proposal. It is important to note that these letters include both the reviewers' concerns and often their recommendations for addressing these concerns. From these review letters, the major concerns and recommendations regarding research design identified by the reviewers were categorized by using standard methods of content analysis.22 All scientific reviews were analyzed initially by D.R.B., and a code book of major themes was developed. The entire research team then coded separate subsets of the reviews independently. After review and revision of the coding scheme, the entire sample was then recoded and verified by the entire research team. These data were not quantified, because the purpose of the research was to identify the full range of issues that investigators must address to conduct valid and reliable CAM clinical trials.
RESULTS
We identified a total of 46 CAM proposals submitted by the CCOP research bases during the period October 1998 through December 2003. Studies were categorized as complex natural products (n = 16), nutritional therapeutics (n = 15), mind-body interventions (n = 12), and alternative medical systems (n = 3; Table 3). No proposals were submitted to test energy therapies (eg, therapeutic touch, Qigong) or manipulative and body-based methods (eg, massage techniques). Of the total 46 proposed CAM studies, 20 trials (22% of the current CCOP symptom-management portfolio) were active at the time of this research (2004). The remaining 26 were closed, under revision, or not approved.
In our analysis of the review letters, the most challenging research-design issues stemmed from the fact that CAM interventions are available over the counter without regulatory oversight or the same quality controls that govern the distribution of pharmaceutical agents. Literature reviews frequently were faulted for failing to ascertain the precise CAM process or product reported in previous studies; these therapies cannot be assumed to be equivalent, because CAM therapies are largely unregulated. Study rationales that establish preliminary efficacy, proper dosage, and safety were generally inadequate, because the preclinical data that typically accompany the development of new pharmaceutical agents were unavailable. Proposals often lacked an explicit quality-control plan from an identified manufacturer or trainer (for mind-body interventions), which is necessary to ensure consistency and comparability. In addition, studies lacked procedures to control for potential drop-ins and sample contamination, which is a particular concern because CAM therapies are readily available in the marketplace. Finally, many investigators did not provide an adequate justification beyond claiming that patients were already using a CAM therapy for some particular purpose. Investigators did not always specify a plausible hypothesis based on previous research or an explanation of the mechanism of action.
Five major concerns regarding the characteristics of the intervention itself were identified: (1) study justification; (2) standardization and quality control of the intervention; (3) appropriate dose; (4) adverse effects and interactions with other drugs; and (5) the burden of the intervention on patients and health care providers (Table 4). In addition, we identified a number of measurement and design issues. These issues are common to all types of research investigations, but presented unusual challenges caused by the unique characteristics of CAM interventions. Measurement and design issues included (1) identifying an appropriate comparison group, (2) identifying appropriate end points, (3) justifying the anticipated effect sizes used to determine sample size, (4) identifying an appropriate timeline for assessments, (5) subject and clinician blinding to the treatment arm, and (6) sample contamination caused by drop-ins. We describe the intervention issues first, organized by CAM modality, followed by a discussion of measurement and design issues.
Nutritional Therapeutics and Complex Natural Products
Because the implications for study design are similar, we combined nutritional therapeutics and complex natural products as ingestible substances taken for therapeutic intent. Together, they were the most common CAM intervention modality in the CCOP proposals, accounting for more than two thirds of the CAM protocols.
Starting with study justification, many proposals failed to provide sufficient data to justify the conduct of a phase III trial. For example, reviewers questioned whether there were sufficient preliminary data to indicate that gingko biloba would improve cognitive functioning, that ginger would reduce nausea and vomiting, or that St John's wort would reduce hot flashes. Justification of the proposed dose was also deficient in many proposals in this category.
In the review letters, the most significant concern with this CAM modality was that, unlike pharmaceutical agents, ingestible CAM products are not manufactured in ways that guarantee that they are unadulterated and identical to one another. Lack of regulation made the interpretation of previous studies difficult and presented atypical concerns regarding product consistency (eg, concentration of active ingredients) and contamination. The lack of standardization belied the assumption that products evaluated in one trial were the same as those tested in another or that the results using one preparation apply to another preparation. For example, because several different ginger-extract products have been studied, researchers were encouraged to use that product for which there was the most information to build on the results of previous research and strengthen the study justification.
Given consistent concerns about standardization, researchers were uniformly asked to specify one formulation of a product (eg, powder v pill), one species of herb versus another, and one source for obtaining the product. Before approval, all proposals to evaluate these products were required to identify the manufacturer, who had to provide assurances of standardized doses, information about stability of the product, and any special storage conditions that would be needed during the trial. If possible, researchers were encouraged to use a single lot of the agent throughout the trial. Researchers were also asked to consider independent analysis of the product for contamination, especially for pesticides or heavy metals. (One case in point from a non-CCOP study, the herbal dietary supplement PC SPES was found to contain the undeclared prescription drug ingredients warfarin and DES23; thus, these products may have effects unrelated to the CAM agent under investigation.) Researchers were also advised to consider conducting studies of biologic products under an investigator-initiated investigational new drug application with the US Food and Drug Administration.
Many protocols were criticized for lack of data on potential adverse effects of nutritional products such as carnitine, creatine, or coenzyme Q10 and, more specifically, the potential interactions between these agents and chemotherapy. Researchers were asked to provide as much information as possible about anticipated toxicities and define grades of toxicities that would trigger discontinuation. Concerns were raised that many nutritional therapeutic interventions may be contraindicated for patients with pre-existing medical conditions (eg, bleeding disorders or patients taking warfarin or heparin24). Researchers were cautioned that certain agents may even protect tumors by interfering with the metabolism of chemotherapeutic agents.25 St John's wort, for example, has been found to interfere with irinotecan metabolism.26
Mind-Body Interventions
Mind-body interventions (n = 12; Table 3) ranged from relaxation techniques and stress management to various exercise programs, expressive writing, and support-group therapy. Although investigators often cited a fairly large literature on this intervention modality to support the study justification, reviewers raised concerns about their interpretation of previous studies. They cautioned that various stress-management, exercise, and writing interventions reported in the literature could not be considered equivalent and interchangeable. Reviewers of mind-body proposals cited major concerns with standardization, justification of the identified dose, and schedule and length of treatments.
One particular concern of the reviewers was that such "operator-dependent" therapies lacked sufficient discussion of procedures to be used to ensure standardization and quality control, a problem that perhaps was even greater in this domain than in nutritional products. Proposals needed to specify the training and credentials of the mind-body therapist. Research protocols frequently failed to demonstrate adequate training procedures to ensure uniform interventions throughout the trial, discuss requirements for ongoing training, and outline how these processes would be monitored to assure adherence. Reviewers were concerned, for instance, that some nurses would be more skilled in delivering mindfulness relaxation techniques than others; proposed study designs often did not discuss how they would address this potential variability. Similarly, researchers were asked to demonstrate how behavioral interventions would be standardized across different providers and intervention sites. A related problem in mind-body interventions was an inadequate description of procedures to monitor different doses, for example, length of time spent writing each week or intensity and duration of exercise.
Reviewers were also concerned about the potential for adverse events with many mind-body interventions, particularly exercise programs or guided imagery (eg, dissociative experiences).27 Researchers were asked to provide information on the suitability and safety of the interventions for the study population (eg, potential interactions between exercise interventions and treatment-related chemotherapeutic toxicities including fatigue, nausea, vomiting, and diarrhea). Researchers also were requested to submit plans for modifying an exercise program to adjust for the physical abilities and limitations of patients and to provide plans for the analysis that take into account potential variations in dosage.
One particular concern with mind-body interventions was that many are highly labor intensive. The problem was that the time burden might interfere with the willingness of staff and patients to participate, thus limiting general clinical applicability. Many mindfulness relaxation techniques also required extensive staff training. Hence, proposals were directed to demonstrate that sufficient numbers of staff with adequate training were available to carry out the study and that patients would not be overly burdened. Proposals also needed to address whether the nurse (or other practitioner) and patient would have sufficient time to develop an effective relationship in conducting mindfulness relaxation exercises and other similar procedures.
Alternative Medical Systems
In the category of alternative medical systems, the CCOP investigators submitted proposals to study acupressure, acustimulation, and a homeopathic intervention, Traumeel S. Because acupressure and acustimulation are taught to patients by trained staff and administered by the patients themselves, standardization and quality control were major concerns identified by reviewers. Concerns about standardization focused on the need to specify a precise and uniform amount of the intervention: the frequency at which the acupressure device would be set; number of times per day and per week that the treatment would be administered; precise site of stimulation; and support for these decisions with data. In terms of quality control, proposals were required to demonstrate that the training plan for staff and patients was adequate to prepare them to administer the intervention reliably over time. Regarding potential adverse effects, reviewers' concerns centered on the need to specify procedures for adjusting the dose if subjects experienced toxicity (eg, nausea and vomiting).
For homeopathic interventions, the most significant issue was study justification: providing empirical data to explain why the proposed intervention could be expected to be effective. For quality control purposes, researchers were asked to address questions about shelf life and storage conditions to ensure consistency of the intervention over time and to provide a plan to assess patient compliance with the treatment regimen. The practice of homeopathy is based on its proposed "law of similars," which suggests that "like cures like." That is, a substance that causes specific symptoms in a healthy person could cure those same symptoms in a sick person. Homeopathic medicines are intended to enable the body to initiate the healing process rather than eliminate the symptoms. Homeopathic medicines are traditionally very small doses of natural substances that have been diluted in water and/or alcohol many times.
Measurement and Design Issues
Most measurement and design issues identified in our analysis are common to all clinical trials of symptom-management interventions but raise particular challenges because of the unusual characteristics of CAM interventions. A recurring concern was identifying an appropriate control intervention. For example, to rule out potential placebo effects, interactive interventions (eg, relaxation techniques, acupressure, etc) needed to specify a control intervention that differed from the study intervention only in the essential elements addressed in study hypotheses but not in other aspects such as attention from study personnel or time spent on the procedure. Examples were the use of a sham acupressure technique28 to allow the evaluation of the correct placement or the use of relaxing music as a comparison condition for mindfulness relaxation. Evaluation of complex natural products that have distinctive odors (such as ginger) presented yet another challenge in identifying an appropriate comparison group. On the one hand, reviewers were concerned that a control intervention that had no odor might elicit suspicion from control participants. Conversely, a control intervention that had a distinctive odor might exaggerate any potential placebo effects. Investigators were encouraged to use products that have no odor when available.
Mind-body interventions also presented particular challenges in determining appropriate end points. An example was whether a mindfulness relaxation technique directly affects the symptoms (eg, number of incidents of vomiting) or merely affects the subject's perceptions of symptoms (eg, how bothersome they are). Other questions included whether a particular product (such as coenzyme Q10) was intended to prevent or relieve fatigue or whether a product (such as valerian) was hypothesized to impact both sleep quality and fatigue or one but not the other.
In the absence of substantial preliminary data regarding the onset and duration of the effects (eg, of acupressure or homeopathic preparations), study designs often failed to specify and justify an optimal time for conducting follow-up assessment. Similarly, in conducting power analyses to compute the sample size, the stated effect size often needed additional justification.
In proposals for which the same staff are responsible for conducting comparison interventions, researchers had to address how they would prevent contamination to ensure, for example, that any mindfulness relaxation techniques did not inadvertently spill over into interactions with patients in the comparison group.
Finally, because many CAM products and procedures are available outside the medical setting, another major concern was the problem of "drop-ins" and sample contamination. Investigators were asked to develop a careful plan to monitor whether patients might seek out CAM products or practices on their own.
DISCUSSION
Explanations for the growth in CAM use include marketing forces, the desire of patients to be actively involved in medical decision making, and dissatisfaction with conventional Western medicine.4 Most CAM therapies, however, have not been tested rigorously to determine their efficacy in reducing symptoms. Worse, little is known about their safety in combination with conventional cancer therapies. Despite wide public acceptance, it is critically important to review the scientific literature on these products, carefully assess risk-benefit ratios, and design research investigations appropriate to current levels of knowledge.
One finding of this research, the predominance of interventions using ingestible substances over other CAM modalities, deserves comment. Because the list of CAM interventions listed in Table 3 consists of all CAM studies submitted by the CCOP to the NCI and does not reflect the effects of the NCI review, the distribution of studies reflects the investigators' interests and their internal review processes. Although we have no data to support the assumption, CCOP investigators are clinicians trained in Western medicine and may be more comfortable with CAM interventions that resemble medical practices used in their training and experience. This conjecture points to a limitation of the research. Because the study sample was limited to investigations initiated by the CCOP, it does not represent the universe of CAM research, as reflected in the skewed distribution of studies across the different CAM modalities. To increase the range of CAM studies (eg, in energy therapies or manipulative methods), investigators may need to form partnerships with experienced CAM practitioners to explore their potential for cancer-symptom relief. Such partnerships would undoubtedly raise new research-design issues. It is worth noting, for example, that this sample of studies did not propose the use of any CAM inclusion or outcome criteria such as the patient's vital force, pulse or tongue diagnoses, or suggestibility in mind-body interventions, to name but a few. The operationalization and measurement of these concepts would present new challenges but might open the field to new discoveries as well.
A number of research-design issues that need to be addressed to advance the state of the science on CAM interventions have been identified with the results of this research. The most significant concerns raised by the reviewers focused on the need to (1) justify the proposed intervention, specifying plausible data-based hypotheses regarding the clinical effects of the proposed intervention, (2) standardize the intervention by gaining agreements with the manufacturers beforehand and implementing appropriate quality-control procedures during the course of the trial, and (3) monitor the comparison group to avoid sample contamination. Although the results of similar studies (eg, effect of a CAM product on fatigue in patients with heart disease) provide grounds for formulating a plausible hypothesis, proposals could be strengthened greatly by specifying the active ingredient and hypothesized mechanism of action. Because the usual preclinical data found in research on new pharmaceutical agents are generally not available for many CAM interventions, researchers may want to consider conducting pilot studies before moving into phase III efficacy trials.
Because "natural" products may have many active ingredients, with varying concentrations between products and even between different batches of the same product, they cannot be assumed to be safe simply because they are already in widespread use. The physiologic and pharmacologic implications of administering these products to ill patients who are also receiving other medications must always be considered carefully. As with other new pharmaceutical agents, researchers should contact the US Food and Drug Administration regarding the need for an investigational new drug application for any ingestible CAM product.
This report highlights many of the complexities of conducting rigorous scientific research on CAM products and practices. Although the issues raised illustrate how research on CAM interventions may be more difficult than studies of new pharmaceutical agents, it is important for future research protocols to address these complexities directly. Stronger research designs are essential to evaluate CAM interventions rigorously and influence clinical practice and general public awareness.
Authors' Disclosures of Potential Conflicts of Interest
The authors indicated no potential conflicts of interest.
NOTES
Authors' disclosures of potential conflicts of interest are found at the end of this article.
REFERENCES
Ni H, Simile C, Hardy AM: Utilization of complementary and alternative medicine by United States adults: Results from the 1999 National Health Interview Survey. Med Care 40:353-358, 2002
Wetzel M, Kaptchuk T, Haramati A, et al: Complementary and alternative medical therapies: Implications for medical education. Ann Intern Med 138:191-197, 2003
Eisenberg D, David R, Ettner S, et al: Trends in alternative medicine use in the United States, 1990-1997. JAMA 289:1569-1575, 1998
Barnes PM, Powell-Griner E, McFann K, et al: Complementary and alternative medicine use among adults: United States 2002. Adv Data 343:1-20, 2004
Sparber A, Wootton JC: Surveys of complementary and alternative medicine: Part II. Use of alternative and complementary cancer therapies. J Altern Complement Med 7:281-287, 2001
Metz J, Jones H, Devine P, et al: Cancer patients use unconventional medical therapies far more frequently than standard history and physical examination suggest. Cancer J 7:149-154, 2001
Jones H, Metz J, Devine P, et al: Rates of unconventional medical therapy use in patients with prostate cancer: Standard history versus directed questions. Urology 59:272-276, 2002
Kao D, Devine P: Use of complementary health practices by prostate carcinoma patients undergoing radiation therapy. Cancer 88:615-619, 2000
Miller M, Boyer M, Butow P, et al: The use of unproven methods of treatment by cancer patients: Frequency, expectations and cost. Support Care Cancer 6:337-347, 1998
Richardson M, Sanders T, Palmer J, et al: Complementary/alternative medicine use in a comprehensive cancer center and the implications for oncology. J Clin Oncol 18:2505-2521, 2000
Ott MJ: Complementary and alternative therapies in cancer symptom management. Cancer Pract 10:162-166, 2002
Paltiel O, Avitzour M, Peretz T, et al: Determinants of the use of complementary therapies by patients with cancer. J Clin Oncol 19:2439-2448, 2001
Burstein H, Gelber S, Guadagnoli E, et al: Use of alternative medicine by women with early-stage breast cancer. N Engl J Med 340:1733-1739, 1999
National Cancer Institute: Cancer research portfolio. Available at: http://researchportfolio.cancer.gov.
White JD: Complementary, alternative, and unproven methods of cancer treatment, in DeVita VT, Hellman S, Rosenberg SA (eds): Cancer: Principles and Practice of Oncology (6th ed). Philadelphia, PA, Lippincott, Williams & Wilkins, 2001, pp 3147-3157
National Cancer Institute, Office of Cancer Complementary and Alternative Medicine: Understanding CAM. Available at: http://cancer.gov/cam/health_understanding.html
Office of Cancer Complementary and Alternative Medicine: Expert Panels in Cancer CAM Research: Developing the State of the Science in Research Methodologies. Bethesda, MD, Department of Health and Human Services, National Institutes of Health, National Cancer Institute, DHHS publication 03-5427, 2003
Smith W: Research methodology: Implications for CAM pain research. Clin J Pain 20:3-7, 2004
Schaffner KF: Assessments of efficacy in biomedicine: The turn toward methodological pluralism, in Callahan D (ed): The Role of Complementary and Alternative Medicine: Accommodating Pluralism. Washington, DC, Georgetown University Press, 2002
Hufford D: CAM and cultural diversity: Ethics and epistemology converge, in Callahan D (ed): The Role of Complementary and Alternative Medicine: Accommodating Pluralism. Washington, DC, Georgetown University Press, 2002
O'Connor B: Personal experience, popular epistemology, and complementary and alternative medicine research, in Callahan D (ed): The Role of Complementary and Alternative Medicine: Accommodating Pluralism. Washington, DC, Georgetown University Press, 2002
Miles M, Huberman A: Qualitative Data Analysis: An Expanded Sourcebook (2nd ed). Thousand Oaks, CA, Sage Publications, 1994
White J: PC-SPES: A lesson for future dietary supplement research. J Natl Cancer Inst 94:1261-1262, 2002
Dasgupta A: Review of abnormal laboratory test results and toxic effects due to use of herbal medicines. Am J Clin Pathol 120:127-137, 2003
Sparreboom A, Cox MC, Acharya MR, et al: Herbal remedies in the United States: Potential adverse interactions with anticancer agents. J Clin Oncol 22:2489-2503, 2004
Mathijssen RH, Verweij J, de Bruijn P, et al: Effects of St. John's wort on irinotecan metabolism. J Natl Cancer Inst 94:1247-1249, 2002
Persinger MA: Enhanced incidence of "the sensed presence" in people who have learned to meditate: Support for the right hemispheric intrusion hypothesis. Percept Mot Skills 75:1308-1310, 1992 (3 pt 2)
Kaptchuk TJ, Berman B, Mausell RB: The placebo and complementary and alternative medicine: Historical, scientific, and methodological considerations, in Office of Cancer Complementary and Alternative Medicine (ed): Expert Panels in Cancer CAM Research: Developing the State of the Science in Research Methodologies. Bethesda, MD, Department of Health and Human Services, National Institutes of Health, National Cancer Institute, DHHS publication 03-5427, 2003(David R. Buchanan, Jeffre)