当前位置: 首页 > 期刊 > 《临床肿瘤学》 > 2005年第7期 > 正文
编号:11329355
Agreement Between Self-Reported Breast Cancer Treatment and Medical Records in a Population-Based Breast Cancer Family Registry
http://www.100md.com 《临床肿瘤学》
     the Peter MacCallum Cancer Centre

    The University of Melbourne

    The Cancer Council Victoria, Victoria, Australia

    Prince of Wales Hospital, NSW, Australia

    Huntsman Cancer Institute, Salt Lake City, UT

    University of Otago, Dunedin, New Zealand

    ABSTRACT

    PURPOSE: Although self-report data on treatment for breast cancer are collected in some large epidemiologic studies, their accuracy is unknown.

    METHODS: As part of a population-based Breast Cancer Family Registry, questionnaires on initial breast cancer treatment and subsequent recurrence were mailed to Australian women diagnosed between 1991 and 1998. These self-report data were validated against medical records for 895 women.

    RESULTS: The median recall period was 3.2 years, mean age at diagnosis was 44 years, and 81% of women had early-stage breast cancer. Agreement between the two data sources was very high for general questions about type of treatment (100%, 99%, 99%, and 94% for surgery, radiotherapy, chemotherapy, hormonal therapy, respectively). For more specific questions about details of each treatment received, agreement was: for radiation therapy, 96% and 99% for radiation to the breast and chest wall, respectively; for surgery, 83%, 97%, and 88% for lumpectomy, mastectomy, and lymph node dissection, respectively; for hormonal therapy, 94% for tamoxifen; and for chemotherapy, range between 76% and 93%. There was 97% agreement about whether there had been a recurrence, and agreement about the location of recurrence was at least 90% for all sites. Agreement regarding stage at diagnosis was 62%, with discrepancies mostly due to women with locoregional disease incorrectly reporting distant spread.

    CONCLUSION: This self-report questionnaire can be used to collect accurate data on broad categories of initial breast cancer treatment and recurrence, and even for more detailed information on specifics of treatment and site of recurrence.

    INTRODUCTION

    Abstraction of data from medical records is generally considered the most accurate means of obtaining information regarding cancer treatment and outcomes. This method of data collection can be time-consuming and expensive, particularly in settings such as large cancer epidemiology studies in which patient treatment and follow-up are decentralized and may involve input from multiple disciplines. In such circumstances, data on each patient may need to be abstracted from the separate medical records of a number of clinicians, including at least one surgeon, medical oncologist, and radiation oncologist, each potentially at different centers. In addition, for prospective studies, the medical records may need to be abstracted on multiple occasions over time to update data on treatment and outcomes.

    Self-report questionnaires are a much cheaper way of obtaining data and have been used extensively in epidemiological studies. Results of validation studies that have compared self-report questionnaires with medical records have variously found low to high agreement between the two data sources, depending on the data being collected and the study setting.1-6 Only a few studies have assessed the accuracy of self-reports of cancer treatments and outcomes,7-9 and to the best of our knowledge, none have addressed breast cancer.

    The National Institutes of Health–funded Breast Cancer Family Registry (CFR) is an international consortium established in 1995 as a resource for research on the epidemiological, clinical, and genetic aspects of breast cancer.10 The Breast CFR has recruited, and is following, more than 12,000 population-based and clinic-based case families from the Unites States, Canada, and Australia and has been collecting data using the same questionnaires at each study site. In particular, the Breast CFR developed a self-report questionnaire on breast cancer treatment and recurrence. Here we report on a validation study of the questionnaire against medical record data using the Australian Breast Cancer Family Study (ABCFS), a population-based component of the Breast CFR.

    METHODS

    Subjects

    Subjects were participants in the ABCFS, a population-based case-control-family study of the genetic, environmental and lifestyle factors associated with breast cancer.11,12 The study commenced in 1992 and has been additionally supported as part of the Breast CFR since 1996. The ABCFS recruited women who had incident primary breast cancer and were living in Melbourne or Sydney, Australia. Recruitment was via the respective state Cancer Registries (reporting of cancer to these state registries is a legislative requirement). Approval for the study was obtained from the ethics committees of The University of Melbourne and The Cancer Councils of Victoria and New South Wales. All women provided written informed consent for participation in the ABCFS. The overall participation rate was 69%. Nonparticipation was due to attrition by death (2%), refusal by the attending doctor (8%), refusal by the woman (16%), nonresponse by the attending doctor (1%), nonresponse by the woman (1%), or inability to locate the woman (2%).13 Women enrolled via the New South Wales Cancer Registry before 1996 were excluded from this study because the consent they gave at the time of entry specifically prohibited any further approaches from the researchers. Of the remaining 1,358, 89 were known to be deceased at the time of questionnaire mailout, leaving 1,269 women to whom the treatment questionnaire was mailed. Of these, 41 were subsequently found to be deceased. Of the remaining 1,228 women, 944 (77%) returned a completed self-report questionnaire and consented to access to their medical records. The medical records were able to be accessed and adequate data obtained on 933 (99%) of these women. Based on information in their medical record, 38 women were found to have had more than one primary breast cancer, and were excluded, leaving a total of 895 subjects in this validation study.

    Data Collection

    Data collection at entry into the ABCFS were by interviewer-administered questionnaire and included detailed information on demographics, epidemiologic risk factors, and family history of cancer.13 Specific to this study, in December 1999, eligible living women enrolled in the ABCFS were mailed a self-report questionnaire about the treatment and outcomes of their breast cancer, and were also asked for consent for the researchers to abstract relevant data from their medical records.

    Self-Report Questionnaire

    The self-report questionnaire was developed by one of us (S.B.) for the Breast CFR and piloted on a small sample of breast cancer patients attending clinics at the Huntsman Cancer Institute in Utah (Fig 1). It consists of six items addressing stage, (question 1), the type of initial breast cancer treatments received (questions 2 to 5) and whether the cancer has recurred and if so when and in which sites (question 6). Each item related to treatment and cancer recurrence first asks whether the broad category of treatment/recurrence was received/occurred (eg, "did you have radiation for this breast cancer") and then, for those who respond in the affirmative, more specific details of the treatment or recurrence are asked.

    Medical Record Abstraction

    Two experienced and trained research assistants abstracted the medical records. For most subjects, several different medical records at multiple hospitals and doctors' private clinics had to be accessed. Detailed data were abstracted from the medical records using a data collection form specifically designed for that purpose, and included: menopausal status at diagnosis; type of breast surgery; details of any adjuvant therapy (radiation, chemotherapy, hormonal therapy; type, duration, and dose); information on locoregional recurrence, first distant recurrence, and therapy following recurrence; and death. Five percent of record abstractions by each research assistant were checked for accuracy by one of us (K.A.P.). For 30 subjects, data were abstracted from the medical record independently by both research assistants, and inter-rater reliability for the data relevant to this validation study was 100%.

    Statistical Methods

    Equality of distributions of demographic characteristics by participation status was assessed using Pearson's 2 test. For ordered categorical variables, trends in differences in distributions were assessed using the gamma statistic and its asymptotic SE.14 Agreement of self-reported stage, treatment, and recurrence with information abstracted from the medical records was assessed by calculating both the proportion in agreement and the kappa statistic.14 Positive and negative predictive values were also calculated where appropriate. These summary statistics were reported because each gives different information about the agreement observed. Proportion correct is a percentage that is easily understood and interpreted. The kappa statistic is an alternative measure commonly used in the literature that takes account of agreement due to chance; however, the reported values are not as readily interpretable by the general reader. Both measures implicitly assume that, for a given question, disagreement for one response is of equal importance to that for another response. Positive predictive value and negative predictive value are also easily interpreted proportions for "yes/no" questions that, when interpreted together, allow for the relative importance of false-negatives versus false-positives to be considered. Following the classification system suggested by Landis and Koch,15 self-reported "unknown" was treated as "no" for the purpose of assessing agreement. Associations between accuracy of self-report and potential explanatory factors, including year of diagnosis, education status, marital status, age at diagnosis, country of birth, and presence of disease recurrence at the time of questionnaire administration, were assessed using unconditional logistic regression with responses coded as correct or not based on medical record abstraction. For this latter purpose, self-reported "unknown" was coded as incorrect. Statistical analyses were conducted using Stata 8.0 (Stata Corporation, College Station, TX), and all statistical tests were two-sided.

    RESULTS

    Table 1 shows that the mean age of subjects at the time of their breast cancer diagnosis was 44 years (range, 23 to 69 years), with 41% under the age of 40 years. All subjects were diagnosed with breast cancer between 1991 and 1998, with the majority diagnosed in 1996 or 1997. The median recall period was 3.2 years, with a range of 1.6 to 9 years. Most subjects were born in Australia (69%) and married (70%) at the time of their enrollment in the ABCFS, and about a quarter (23%) were university graduates. The majority had early stage breast cancer (81%), with an almost equal number with stage I (41%) compared with stage II (40%) disease at diagnosis. Subjects who participated in the validation study were more likely to be diagnosed more recently (P = .02), to have been born in Australia (P < .001), and be married (P = .02) compared with those to whom the questionnaire was mailed but who did not participate (Table 1).

    According to their medical records, 543 subjects (61%) had had a lumpectomy either with or without axillary nodal dissection (525 and 18, respectively) as their definitive surgery, 345 (38%) had had mastectomy either with or without axillary nodal dissection (342 and three, respectively), one had not had surgery, and for six, the type of definitive surgery could not be determined from their medical record. Overall, 571 subjects (64%) had adjuvant radiation therapy, 370 (41%) received adjuvant hormonal therapy, and 545 (61%) received adjuvant or neoadjuvant chemotherapy.

    Assuming the medical record as correct, the validation of each question on the self-report questionnaire is summarized in Table 2 in terms of the proportion of subjects for whom the self report agreed with the medical record data (proportion correct), the strength of agreement (), the sensitivity and specificity, and the positive and negative predictive values (PPV and NPV, respectively) for each question.

    For question 1, only 62% of subjects correctly identified the stage of the breast cancer when it was diagnosed (ie, involving breast only, breast and axillary lymph nodes or distant spread). However, the most common discrepancy for this question was 268 subjects incorrectly reporting that their cancer had "spread to other areas of the body besides the breast and lymph glands (nodes) under the arm," but their medical records indicated only locoregional disease. Adjuvant or neoadjuvant chemotherapy had been received by 234 (87%) of these, and 161 (60%) had had adjuvant hormonal therapy, with only three subjects (1%) having had no adjuvant systemic therapy. Subjects were more likely to answer this question correctly the more distantly in time that their cancer had been diagnosed (P < .001) and if they had not had disease recurrence at the time the questionnaire was administered (P = .002).

    All but one subject correctly identified whether or not they had undergone surgery for their breast cancer, with 97% ( = 0.93), 83% ( = 0.57) and 88% ( = 0.26) correctly identifying that they had undergone mastectomy, lumpectomy, and/or axillary dissection, respectively (questions 2 and 2.1). Of the 151 subjects who incorrectly reported not having had a lumpectomy, 128 had a mastectomy after their initial lumpectomy which they correctly reported. Thus, it is possible that these women did not follow the "mark all that apply" instruction for that question, but rather ticked the box that related to their final surgical status of mastectomy. Women who had had a breast cancer recurrence by the time the questionnaire was administered were less likely to correctly report whether they had had a mastectomy as part of their initial treatment (92% correct compared with 97%, P = .003). None of the other factors tested was associated with greater accuracy of self-response for these surgical questions.

    For the broad question about whether there had been radiation given as part of the initial treatment, the agreement between self-report and medical record data was very high, with 99% correct; = 0.97; and sensitivity, specificity, PPV, and NPV all at least 98%. Women who had had a breast cancer recurrence by the time the questionnaire was administered were less likely to answer this question correctly (91% correct v 100% correct, P < .001). Of those who reported they had had radiation, 96% ( = 0.85) and 99% ( = 0.94) of those who received radiation to the breast or chest wall respectively reported it correctly. For women who had undergone radiation to the breast, none of the factors examined was associated with greater accuracy of self-report, but women who were older or who had been diagnosed more recently reported more accurately that they had received chest wall irradiation (P = .001 and P < .008, respectively).

    For the broad question about whether initial hormonal therapy had been received, 94% of subjects self-reported correctly and the degree of agreement was excellent ( = 0.87) with sensitivity, specificity, PPV, and NPV all at least 92%. Women who had had a breast cancer recurrence by the time the questionnaire was administered were less likely to provide accurate answers to this question (82% correct v 94%, P < .001). However, for the subquestion regarding the type of hormonal therapy, agreement between the two data sources regarding whether the agent used was tamoxifen was only moderate ( = 0.47), with 94% of subjects responding correctly and high sensitivity (97%) and PPV (96%) but lower specificity (44%) and NPV (58%). Again, women who had had a breast cancer recurrence before the time the questionnaire was administered were less likely to answer this question correctly (80% correct v 94%, P = .009).

    For the broad question about whether chemotherapy was given as part of the initial breast cancer treatment, 99% of subjects responded correctly, and there was excellent agreement between the data sources ( = 0.98) and sensitivity, specificity, PPV and NPV were all 99%. The self-report questionnaire performed less well in identifying the type of chemotherapy received with only 76%, 88%, and 93% correctly identifying that they had the CMF (cyclophosphamide, methotrexate, fluorouracil), AC (doxorubicin, cyclophosphamide) or EC (epirubicin, cyclophosphamide) regimens respectively ( = 0.44, 0.67, and 0.62). Both for the broad question about whether chemotherapy was received and for the sub-questions regarding the type of chemotherapy received, women were more likely to respond correctly if they were more highly educated and born in Australia (P .001 for all comparisons).

    Agreement between the two data sources with respect to recurrence of the breast cancer was high. Overall 97% of subjects correctly identified whether they had had a recurrence of their breast cancer ( = 0.85), and at least 90% correctly reported the site of their recurrence ( values 0.59 to 0.81). Eighty-two percent (69 of 84) of women provided the correct year of their initial breast cancer recurrence. None of the factors examined was associated with greater accuracy of self-report for these questions.

    DISCUSSION

    This study showed that the agreement between self-report and medical records was very high for the broad questions about type of initial treatment received for breast cancer (ie, surgery and/or radiotherapy, and/or chemotherapy and/or hormonal therapy). For the sub-questions about details of treatment, agreement was also high (greater than 95% correct) for radiation therapy details and although lower for surgical, chemotherapy, and hormonal therapy details, in all instances the proportion correct was at least 75%. There was 97% agreement for whether or not there had been a recurrence, with at least 90% agreement for the sub-questions about where the recurrence had occurred. Agreement was lowest for the extent of the initial breast cancer at diagnosis with only 62% reporting this correctly.

    Women who had already experienced a breast cancer recurrence at the time of administration of the questionnaire were less likely to respond correctly to most questions, apart from the question about chemotherapy treatment. This seemed to mostly be due to women reporting treatment that they had had for recurrent disease, despite the fact that the lead in to the questions specifically stated that they asked about "treatment given for the breast cancer at the time it was first diagnosed," and participants were also specifically asked not to include "treatment given for any cancer that came back after the original treatment." Associations of other subject characteristics with accuracy were not seen for all questions, but where present tended to include higher education level, Australia as the country of birth (which may have been a surrogate for first language), and more recent diagnosis (a surrogate for recall period). Paradoxically, subjects diagnosed more recently were less likely to answer correctly the question about the extent of disease at diagnosis. The explanation may be that these subjects may have been more likely to have had the concept of micrometastases explained to them and this may have affected their responses to this question.

    The results of the study suggest that this self-report questionnaire can be used to collect accurate data on broad categories of initial treatment that women have received for their breast cancer, and whether they have experienced disease recurrence. Even for more detailed information about specifics of treatment, and site of recurrence, data were generally reasonably accurate, but for information on the initial extent of disease at diagnosis this form of self-report is less accurate. It is also less accurate when administered to women who have already had a recurrence of their breast cancer. Future use of the questionnaire should explore ways to strengthen the wording to ensure that women report only their adjuvant treatment and not treatment received after a recurrence. In the meantime this questionnaire can be reliably used to collect adjuvant treatment information on women before the onset of any disease recurrence, thus use of the questionnaire closer to the time of initial adjuvant treatment would be optimal.

    The inability to accurately determine stage of disease at presentation in a substantial minority (38% in this study) may be a limitation for some types of studies for which use of this self-report instrument might be considered. For example, studies of prognosis require accurate data on treatment and outcomes, which this self-report instrument can provide, but also require accurate data on disease stage at diagnosis, which is less reliably provided by this instrument. For such situations, and provided the cost-trade-off was considered justified, mixed methodology for data collection might be considered; that is, collecting treatment and outcome data using self-report but accessing details on stage of disease from either medical records or cancer registries. Such an approach is likely to still be cheaper than collecting all treatment and outcome data by abstraction from medical records, especially if it is done at or around time of recruitment. In our study the treatment questionnaire was not developed until several years into recruitment, and hence we were unable to get self-reported treatment information on all 1,358 patients recruited into the ABCFS cohort because they were either deceased (130 patients) or chose not to return the self-report questionnaire (284 patients). This raises the possibility of selection bias in studies of prognosis, which should be addressed by sensitivity analyses, and carefully considered when interpreting results. It should also be noted that this questionnaire was not designed for use in studies of quality of care, and so does not include variables such as timeliness of diagnosis and treatment, appropriateness of treatment and whether treatment was compatible with patient preferences.

    This validation study was undertaken in a population-based cohort of women and thus one of its strengths is that it was not limited to women treated within a single institution or group of institutions. This study essentially evaluated the validity of the self-report questionnaire in women under age 60 at diagnosis (only 1% were older), so is unable to determine the validity of self-reports of breast cancer treatment in older women. Within the age range of women studied age was not a predictor of correct response.

    The disagreement seen between the self-report data and those extracted from the medical record may have been due to errors in women's recall, wording of the questionnaire items, incomplete data in the medical records, or women's noncompliance with prescribed therapy.

    Subjects were generally more likely to correctly recall broad details of their treatment rather than the specifics, but were more likely to recall treatment details if there were additional prompts. For example, there was very high agreement between the two data sources for details of radiation treatment, and this question included a prompt about the fact that radiation to the chest wall versus breast was dependent on whether there had been prior mastectomy. The agreement for the specifics of chemotherapy treatment ranged from 76% to 93% correct depending on the chemotherapy regimen received. Given the complexity of this information, it is possible that the prompts within the chemotherapy sub-question about type of chemotherapy received may have assisted recall, although that could not be specifically tested.

    Wording of items was likely to have been an explanation for the worse agreement for some questions. For example, it is possible that for the question about the extent of disease at diagnosis, subjects incorrectly reported "spread to other areas of the body besides the breast and lymph glands (nodes) under the arm" because they were aware of the concept of micrometastases. The distinction is complex, and it may be difficult to collect staging data by self-report, regardless of the specific wording of the question.

    In this study, it was assumed that the data abstracted from the medical record were correct. For some treatment details the data contained in the medical record may have been incomplete. For example, 28 subjects reported that they had had hormonal therapy for their cancers, but there was no documentation of hormonal therapy abstracted from the medical record. In some cases a prescription for hormonal therapy may have been written but not documented in the medical record; alternatively, despite the ABCFS attempting to abstract records from all treating clinicians, some may have been missed, particularly office-based clinicians. Noncompliance with treatment may also account for some of the discrepancy between self-report and data in the medical record for hormonal therapy. Another 28 subjects reported they did not take hormonal therapy despite the fact that it was documented as prescribed in the medical record. It is possible that a proportion of these subjects did not take the prescribed hormonal therapy, so their self-report may in fact have been correct. The Prostate Cancer Outcomes Study has highlighted similar issues suggesting that medical record review may not be the gold standard for hormonal therapy.7 A Swedish prospective cohort study of more than 16,000 women confirmed the validity of self-reports of current use of hormone therapy when compared with a personal diary,16 so patient self-report may be more accurate than medical record abstraction for self-administered oral medications.

    To the best of our knowledge, this is the first population-based comparison of self-reported details of treatment and recurrence of breast cancer with medical records. The results demonstrate the validity of this self-report questionnaire developed by the Breast CFR, and the levels of validity found in this Australian setting are likely to apply in North America and other populations with similar standards of breast cancer care. The results should be of use to other researchers who are contemplating using self-report questionnaires rather than medical record abstraction to determine details of breast cancer treatments and recurrences.

    Authors' Disclosures of Potential Conflicts of Interest

    The authors indicated no potential conflicts of interest.

    Acknowledgment

    We thank Pauline Jacklin and Amanda Muir for medical record data abstraction, Gillian Dite and Judi Maskiell for assistance with administering the self-report questionnaire, Maggie Angelakos for data management, and Joanne McKinley for assistance with preparation of the manuscript.

    NOTES

    Supported by the National Health and Medical Research Council of Australia (No. 145604). The Australian Breast Cancer Family Study was supported by the National Health and Medical Research Council of Australia, the New South Wales Cancer Council, the Victorian Health Promotion Foundation and the United States National Cancer Institute, National Institutes of Health under RFA No. CA-95-003 as part of the Breast Cancer Family Registries (CFRs), and through cooperative agreements with the Fox Chase Cancer Center, Huntsman Cancer Institute, Columbia University, Northern California Cancer Center, Cancer Care Ontario, and The University of Melbourne.

    The content of this manuscript does not necessarily reflect the views or policies of the National Cancer Institute or any of the collaborating centers in the Breast CFRs, nor does mention of trade names, commercial products, or organizations imply endorsement by the US Government or the CFRs.

    Authors' disclosures of potential conflicts of interest are found at the end of this article.

    REFERENCES

    Zhu K, McKnight B, Stergachis A, et al: Comparison of self-report data and medical records data: Results from a case-control study on prostate cancer. Int J Epidemiol 28:409-417, 1999

    Bergmann MM, Byers T, Freedman DS, et al: Validity of self-reported diagnoses leading to hospitalization: A comparison of self-reports with hospital records in a prospective study of American adults. Am J Epidemiol 147:969-977, 1998

    Colditz GA, Martin P, Stampfer MJ, et al: Validation of questionnaire information on risk factors and disease outcomes in a prospective cohort study of women. Am J Epidemiol 123:894-900, 1986

    Newell SA, Girgis A, Sanson-Fisher RW, et al: The accuracy of self-reported health behaviours and risk factors relating to cancer and cardiovascular disease in the general population: A critical review. Am J Prev Med 17:211-229, 1999

    Paganini-Hill A, Ross RK: Reliability of recall of drug usage and other health-related information. Am J Epidemiol: 116:114-122, 1982

    Linet MS, Harlow SD, McLachlin JK, et al: A comparison of interview data and medical records for previous medical conditions and surgery. J Clin Epidemiol 42:1207-1213, 1989

    Clegg LX, Potosky AL, Harlan LC, et al: Comparison of self-reported initial treatment with medical records: Results from the Prostate Cancer Outcomes Study. Am J Epidemiol 154:582-587, 2001

    Coulter A, McPherson K, Elliott S, et al: Accuracy of recall of surgical histories: A comparison of postal survey data and general practice records. Community Med 7:186-189, 1985

    Kaden-Lottick NS, Robison LL, Gurney JG, et al: Childhood cancer survivors' knowledge about their past diagnosis and treatment: Childhood Cancer Survivor Study. JAMA 287:1832-1839, 2002

    John EM, Hopper JL, Beck JC, et al: The Breast Cancer Family Registry: An infrastructure for cooperative multinational, interdisciplinary and translational studies of the genetic epidemiology of breast cancer. Breast Cancer Res 6:R375-R389, 2004

    McCredie MR, Dite GS, Giles GG, et al: Breast cancer in Australian women under the age of 40. Cancer Causes Control 9:189-198, 1998

    Hopper JL, Giles GG, McCredie MRE, et al: Background, rationale and protocol for a case-control-family study of breast cancer. The Breast 3:79-86, 1994

    Dite GS, Jenkins MA, Southey MC, et al: Familial risks, early-onset breast cancer, and BRCA1 and BRCA2 germline mutations. J Natl Cancer Inst 95:448-457, 2003

    Agresti A: Categorical Data Analysis (ed 2). New Jersey, John Wiley and Sons Inc, 2002, 88-435

    Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics 33:363-374, 1977

    Merlo J, Berglund G, Wirfalt E, et al: Self-administered questionnaire compared with personal diary for assessment of current use of hormone therapy: An analysis of 16,060 women. Am J Epidemiol 152:788-792, 2000(Kelly-Anne Phillips, Roge)