Accuracy and self correction of information received from an internet breast cancer list: content analysis
1 School of Health Information Sciences, University of Texas Health Science Center at Houston, 7000 Fannin Street, Houston, TX 77030, USA,2 Department of Surgical Oncology, University of Texas MD Anderson Cancer Center, Houston, TX
Abstract
Objectives To determine the prevalence of false or misleading statements in messages posted by internet cancer support groups and whether these statements were identified as false or misleading and corrected by other participants in subsequent postings.
, 百拇医药
Design Analysis of content of postings.
Setting Internet cancer support group Breast Cancer Mailing List.
Main outcome measures Number of false or misleading statements posted from 1 January to 23 April 2005 and whether these were identified and corrected by participants in subsequent postings.
Results 10 of 4600 postings (0.22%) were found to be false or misleading. Of these, seven were identified as false or misleading by other participants and corrected within an average of four hours and 33 minutes (maximum, nine hours and nine minutes).
, 百拇医药
Conclusions Most posted information on breast cancer was accurate. Most false or misleading statements were rapidly corrected by participants in subsequent postings.
Introduction
Nearly half of women recently diagnosed as having breast cancer turned to the internet for information on health.1 Consumers are satisfied with their online experience and are making choices based on the information that they encounter.2 3 In some cases patients may not discuss with clinicians their use of treatments found online.4 On the other hand, clinicians are faced with patients who have been informed (or misinformed) by information posted on the internet. Clinicians, researchers, and healthcare consumers are therefore concerned about the accuracy of online health information.2 5-7
, http://www.100md.com
Internet cancer support groups offer the public a forum in which to share experiences, ask questions, and offer advice. Discussions have been categorised according to their content as related to quality of care, treatment, recurrence of disease, and alternative therapy.8 Rates of false or misleading information may vary depending on the definition of falsehood (for example, verified by multiple observers, possibly or definitely false), forum studied, existence of a moderator, prevalence of health professionals, topic, and other factors. One study found that about 6% of postings to an online epilepsy forum were "objectively inaccurate."9 A review of a German language brain tumour list found that "Normally, precisely formed questions were answered by medically correct and solid statements. When medical statements were incorrect, other participants did not hesitate to correct or criticise."10
, http://www.100md.com
Professionals rely on the peer review process to screen out false or misleading information. Peer reviewed journals ensure that published articles are critically reviewed by experts. Similarly, open source software relies on the user community for quality control. A comparable peer review process may occur on online forums whereby false or misleading information is corrected quickly and reliably by subsequent postings. This possibility, which may be referred to as the "self correction hypothesis," has been suggested in the literature.9 11 12 To our knowledge the self correction hypothesis has never been quantitatively tested in an unmoderated forum. We determined the prevalence of false or misleading statements found on a specific internet cancer support group site and whether these statements were identified and corrected by subsequent postings.
, 百拇医药
Methods
We selected the Breast Cancer Mailing List because it is unmoderated and therefore reflects only the opinion of voluntary participants without any kind of systematic interference from healthcare professionals. The list was formed in 1994 and facilitates communication between people affected by breast cancer. Participants include individuals with breast cancer; their caretakers, family, and friends; and a small number of healthcare professionals. At the time of our analysis, there were about 500 participants to the list, with an average of 310 distinct people uploading messages per month.
, http://www.100md.com
A person can join the Breast Cancer Mailing List by sending an email message to the mailing list administrators. After receiving a message of confirmation, the participant then receives all messages (postings) sent to the list's email address. The participant can post to the list using the same address. The list's archives contain all messages (postings) since its inception. At the time of our study the archives contained over 600 megabytes of information in 248 051 text email messages and were publicly available at http://bclist.petebevin.com/
, http://www.100md.com
We reviewed all messages posted to the list from 1 January to 23 April 2005. A "post" or "posting" was defined as the entire content of a message sent by a participant to the list. We reviewed the entire posting to identify false or misleading statements. A "thread" was defined as a group of postings under one subject. A thread started with the first posting of a new subject and ended when no more postings occurred in response to that subject. The postings in a thread were in chronological order. We excluded from our study postings that were incomplete at the cut-off date (23 April).
, http://www.100md.com
Three independent clinicians reviewed and classified the postings: a general practitioner (AE), a general internist (EVB), and a breast cancer surgeon (FMB). The general practitioner first identified statements that might be false or misleading. Factually incorrect statements and those that were likely to lead a medically naive reader to a false factual conclusion were defined as false and misleading, respectively. For the purposes of this study we did not consider any statement that was phrased as an opinion (I believe that...) or a question to be false or misleading. We did not require an explanation of why the original statement was incorrect to consider the false or misleading statement successfully identified and corrected. However, the correction had to relate to the specific statement that was false or misleading and give the correct facts. We recorded the location of the posting within the thread, the date and time the message was posted, and the number of postings until a participant identified the incorrect information. We considered a false or misleading statement as not identified when we found no evidence of its identification by other participants and the thread was exhausted.
, 百拇医药
EVB and FMB verified the information in the candidate postings identified by AE and validated or challenged the initial judgment. Reviewers EVB and FMB confirmed or dismissed all possibly false or misleading postings, based on review of relevant literature. Differences were resolved by consensus.
Results
Overall, 4600 postings, organised into 1378 threads, were reviewed. Table 1 shows a summary of the total number of postings reviewed per month. AE identified a total of 32 (0.7%) candidate statements containing false or misleading information in 30 threads. Of these 32, 10 (31%) were confirmed to be false or misleading by the other two reviewers on the basis of the relevant literature and consensus among the three independent reviewers. The statements were posted by seven different participants, none of which identified themselves as health professionals. A list of the statements confirmed as false or misleading is on bmj.com.
, http://www.100md.com
What is already known on this topic
Healthcare consumers search the internet for information on health
Online information affects patient's decisions about treatment
Despite the publication of many quality measures, no validated, usable measures exist that can reliably identify false or misleading information online
What this study adds
, 百拇医药
Given a sufficiently active forum, participants can identify and correct most false or misleading statements quickly and reliably without requiring professional review
Online forums can police themselves
Seven of the 10 false or misleading statements were identified and corrected by six participants, three of whom had also posted false or misleading statements. The average time before a false or misleading statement was identified was 4 hours and 33 minutes from when the posting appeared on the forum (range 2h3min-9h9min). For the false or misleading statements that were identified and corrected, the average number of postings before identification was 2 (range 1-3).
, http://www.100md.com
Discussion
Few postings on the internet cancer support group Breast Cancer Mailing List contained false or misleading information. If false or misleading information was posted to the mailing list, it was identified and corrected by a subsequent posting in more than two thirds of cases. Multiple participants posted and corrected false statements, rather than a single expert participant. One individual did, however, post three of the seven corrections that related to a single topic (expiry of drugs).
, 百拇医药
In contrast to our results, one study found that conclusions drawn by authors on a mailing list for painful hand and arm conditions were rarely questioned.13 This finding was not, however, quantified. One possible explanation for this discrepancy is that our study was carried out 11 years later. In 1994, the internet was relatively new and our findings may reflect a maturing medium in which participants are more likely to critically evaluate information. In addition, because participants may have already experienced phases of the disease, they can provide accurate information.14 Perhaps there is more shared experience among patients with breast cancer than among people affected by limb pain. In addition, most of the false or misleading statements that we identified were not likely to lead to harm.
, 百拇医药
Our study was limited by the fact that a single reviewer determined the statements that might be false or misleading. We may therefore have missed some false or misleading statements. For this reason we did not require that a statement be factually incorrect and allowed misleading statements, as defined above. Furthermore, reviewers were not blinded to the study hypotheses. An additional limitation of our study is that we analysed a single, albeit large, internet cancer support group. Our findings may not generalise to other online forums. Unlike most previous studies, however, we chose a large, unmoderated list that truly reflects self correction, rather than the knowledge of a moderator. More research is needed to determine if our findings generalise beyond the Breast Cancer Mailing List to other online communities and other health topics.
, 百拇医药
Ideally consumers would have access to accurate online information without direct professional guidance, so that the limited time they have with clinicians could be used more efficiently. This requires that online resources present accurate information. At this time, no known effective strategies exist to ensure that online information is accurate. Our findings suggest that, given a forum, the internet can police itself.
Editorial by Jadad et al
, http://www.100md.com
Details of false or misleading statements are on bmj.com
This article was posted on bmj.com on 2 March 2006: http://bmj.com/cgi/doi/10.1136/bmj.38753.524201.7C
Contributors: AE, FM-B, and EVB planned the study, collected the data, and drafted and revised the manuscript. EVB is guarantor.
Funding: Supported in part by US National Library of Medicine (grant No 5K22LM008306: to EVB) and the National Cancer Institute (grant No 1K08CA91895: to FM-B.). All authors are independent of the funding agencies.
, http://www.100md.com
Competing interests: None declared.
Ethical approval: Not required.
References
Satterlund MJ, McCaul KD, Sandgren AK. Information gathering over time by breast cancer patients. J Med Internet Res 2003;5: e15.
Fox S, Rainie L. The online health care revolution: how the web helps Americans take better care of themselves. Washington DC: Pew Internet and American Life Project: Online, 2000: 3-7.
, 百拇医药
Hope and the media in advanced cancer patients. American Society of Clinical Oncology 36th annual meeting; New Orleans, 2000.
Eisenberg DM, Kessler RC, Rompay MIV, Kaptchuk TJ, Wilkey SA, Appel S, et al. Perceptions about complementary therapies relative to conventional therapies among adults who use both: results from a national survey. Ann Intern Med 2001;135: 344-51.
Biermann JS, Golladay GJ, Greenfield ML, Baker LH. Evaluation of cancer information on the Internet [see comments]. Cancer 1999;86: 381-90.
, 百拇医药
Jadad AR, Gagliardi A. Rating health information on the internet: navigating to knowledge or to Babel JAMA 1998;279: 611-4.
Price SL, Hersh WR. Filtering web pages for quality indicators: an empirical approach to finding high quality consumer health information on the world wide web. Proc AMIA Symp 1999: 911-5.
Weinberg N, Schmale J, Uken J, Wessel K. Online help: cancer patients participate in a computer-mediated support group. Health Soc Work 1996;21: 24-9.
, 百拇医药
Hoch DB, Norris D, Lester LE, Marcus AD. Information exchange in an epilepsy forum on the world wide web. Seizure 1999;8: 30-4.
Mursch K, Behnke-Mursch J. Internet-based interaction among brain tumour patients. Analysis of a medical mailing list. Zentralbl Neurochir 2003;64: 71-5.
Ferguson T. Heatlh online: how to find health information, support groups and self help communities in cyberspace. Reading, MA: Addison-Wesley, 1996.
, 百拇医药
Feenberg AL, Licht JM, Kane KP, Moran K, Smith RA. The online patient meeting. J Neurol Sci 1996;139(Suppl): 129-31.
Culver JD, Gerr F, Frumkin H. Medical information on the internet: a study of an electronic bulletin board. J Gen Intern Med 1997;12: 466-70.
Till JE. Evaluation of support groups for women with breast cancer: importance of the navigator role. Health Qual Life Outcomes 2003;1: 16., http://www.100md.com(Adol Esquivel, Funda Meri)
Abstract
Objectives To determine the prevalence of false or misleading statements in messages posted by internet cancer support groups and whether these statements were identified as false or misleading and corrected by other participants in subsequent postings.
, 百拇医药
Design Analysis of content of postings.
Setting Internet cancer support group Breast Cancer Mailing List.
Main outcome measures Number of false or misleading statements posted from 1 January to 23 April 2005 and whether these were identified and corrected by participants in subsequent postings.
Results 10 of 4600 postings (0.22%) were found to be false or misleading. Of these, seven were identified as false or misleading by other participants and corrected within an average of four hours and 33 minutes (maximum, nine hours and nine minutes).
, 百拇医药
Conclusions Most posted information on breast cancer was accurate. Most false or misleading statements were rapidly corrected by participants in subsequent postings.
Introduction
Nearly half of women recently diagnosed as having breast cancer turned to the internet for information on health.1 Consumers are satisfied with their online experience and are making choices based on the information that they encounter.2 3 In some cases patients may not discuss with clinicians their use of treatments found online.4 On the other hand, clinicians are faced with patients who have been informed (or misinformed) by information posted on the internet. Clinicians, researchers, and healthcare consumers are therefore concerned about the accuracy of online health information.2 5-7
, http://www.100md.com
Internet cancer support groups offer the public a forum in which to share experiences, ask questions, and offer advice. Discussions have been categorised according to their content as related to quality of care, treatment, recurrence of disease, and alternative therapy.8 Rates of false or misleading information may vary depending on the definition of falsehood (for example, verified by multiple observers, possibly or definitely false), forum studied, existence of a moderator, prevalence of health professionals, topic, and other factors. One study found that about 6% of postings to an online epilepsy forum were "objectively inaccurate."9 A review of a German language brain tumour list found that "Normally, precisely formed questions were answered by medically correct and solid statements. When medical statements were incorrect, other participants did not hesitate to correct or criticise."10
, http://www.100md.com
Professionals rely on the peer review process to screen out false or misleading information. Peer reviewed journals ensure that published articles are critically reviewed by experts. Similarly, open source software relies on the user community for quality control. A comparable peer review process may occur on online forums whereby false or misleading information is corrected quickly and reliably by subsequent postings. This possibility, which may be referred to as the "self correction hypothesis," has been suggested in the literature.9 11 12 To our knowledge the self correction hypothesis has never been quantitatively tested in an unmoderated forum. We determined the prevalence of false or misleading statements found on a specific internet cancer support group site and whether these statements were identified and corrected by subsequent postings.
, 百拇医药
Methods
We selected the Breast Cancer Mailing List because it is unmoderated and therefore reflects only the opinion of voluntary participants without any kind of systematic interference from healthcare professionals. The list was formed in 1994 and facilitates communication between people affected by breast cancer. Participants include individuals with breast cancer; their caretakers, family, and friends; and a small number of healthcare professionals. At the time of our analysis, there were about 500 participants to the list, with an average of 310 distinct people uploading messages per month.
, http://www.100md.com
A person can join the Breast Cancer Mailing List by sending an email message to the mailing list administrators. After receiving a message of confirmation, the participant then receives all messages (postings) sent to the list's email address. The participant can post to the list using the same address. The list's archives contain all messages (postings) since its inception. At the time of our study the archives contained over 600 megabytes of information in 248 051 text email messages and were publicly available at http://bclist.petebevin.com/
, http://www.100md.com
We reviewed all messages posted to the list from 1 January to 23 April 2005. A "post" or "posting" was defined as the entire content of a message sent by a participant to the list. We reviewed the entire posting to identify false or misleading statements. A "thread" was defined as a group of postings under one subject. A thread started with the first posting of a new subject and ended when no more postings occurred in response to that subject. The postings in a thread were in chronological order. We excluded from our study postings that were incomplete at the cut-off date (23 April).
, http://www.100md.com
Three independent clinicians reviewed and classified the postings: a general practitioner (AE), a general internist (EVB), and a breast cancer surgeon (FMB). The general practitioner first identified statements that might be false or misleading. Factually incorrect statements and those that were likely to lead a medically naive reader to a false factual conclusion were defined as false and misleading, respectively. For the purposes of this study we did not consider any statement that was phrased as an opinion (I believe that...) or a question to be false or misleading. We did not require an explanation of why the original statement was incorrect to consider the false or misleading statement successfully identified and corrected. However, the correction had to relate to the specific statement that was false or misleading and give the correct facts. We recorded the location of the posting within the thread, the date and time the message was posted, and the number of postings until a participant identified the incorrect information. We considered a false or misleading statement as not identified when we found no evidence of its identification by other participants and the thread was exhausted.
, 百拇医药
EVB and FMB verified the information in the candidate postings identified by AE and validated or challenged the initial judgment. Reviewers EVB and FMB confirmed or dismissed all possibly false or misleading postings, based on review of relevant literature. Differences were resolved by consensus.
Results
Overall, 4600 postings, organised into 1378 threads, were reviewed. Table 1 shows a summary of the total number of postings reviewed per month. AE identified a total of 32 (0.7%) candidate statements containing false or misleading information in 30 threads. Of these 32, 10 (31%) were confirmed to be false or misleading by the other two reviewers on the basis of the relevant literature and consensus among the three independent reviewers. The statements were posted by seven different participants, none of which identified themselves as health professionals. A list of the statements confirmed as false or misleading is on bmj.com.
, http://www.100md.com
What is already known on this topic
Healthcare consumers search the internet for information on health
Online information affects patient's decisions about treatment
Despite the publication of many quality measures, no validated, usable measures exist that can reliably identify false or misleading information online
What this study adds
, 百拇医药
Given a sufficiently active forum, participants can identify and correct most false or misleading statements quickly and reliably without requiring professional review
Online forums can police themselves
Seven of the 10 false or misleading statements were identified and corrected by six participants, three of whom had also posted false or misleading statements. The average time before a false or misleading statement was identified was 4 hours and 33 minutes from when the posting appeared on the forum (range 2h3min-9h9min). For the false or misleading statements that were identified and corrected, the average number of postings before identification was 2 (range 1-3).
, http://www.100md.com
Discussion
Few postings on the internet cancer support group Breast Cancer Mailing List contained false or misleading information. If false or misleading information was posted to the mailing list, it was identified and corrected by a subsequent posting in more than two thirds of cases. Multiple participants posted and corrected false statements, rather than a single expert participant. One individual did, however, post three of the seven corrections that related to a single topic (expiry of drugs).
, 百拇医药
In contrast to our results, one study found that conclusions drawn by authors on a mailing list for painful hand and arm conditions were rarely questioned.13 This finding was not, however, quantified. One possible explanation for this discrepancy is that our study was carried out 11 years later. In 1994, the internet was relatively new and our findings may reflect a maturing medium in which participants are more likely to critically evaluate information. In addition, because participants may have already experienced phases of the disease, they can provide accurate information.14 Perhaps there is more shared experience among patients with breast cancer than among people affected by limb pain. In addition, most of the false or misleading statements that we identified were not likely to lead to harm.
, 百拇医药
Our study was limited by the fact that a single reviewer determined the statements that might be false or misleading. We may therefore have missed some false or misleading statements. For this reason we did not require that a statement be factually incorrect and allowed misleading statements, as defined above. Furthermore, reviewers were not blinded to the study hypotheses. An additional limitation of our study is that we analysed a single, albeit large, internet cancer support group. Our findings may not generalise to other online forums. Unlike most previous studies, however, we chose a large, unmoderated list that truly reflects self correction, rather than the knowledge of a moderator. More research is needed to determine if our findings generalise beyond the Breast Cancer Mailing List to other online communities and other health topics.
, 百拇医药
Ideally consumers would have access to accurate online information without direct professional guidance, so that the limited time they have with clinicians could be used more efficiently. This requires that online resources present accurate information. At this time, no known effective strategies exist to ensure that online information is accurate. Our findings suggest that, given a forum, the internet can police itself.
Editorial by Jadad et al
, http://www.100md.com
Details of false or misleading statements are on bmj.com
This article was posted on bmj.com on 2 March 2006: http://bmj.com/cgi/doi/10.1136/bmj.38753.524201.7C
Contributors: AE, FM-B, and EVB planned the study, collected the data, and drafted and revised the manuscript. EVB is guarantor.
Funding: Supported in part by US National Library of Medicine (grant No 5K22LM008306: to EVB) and the National Cancer Institute (grant No 1K08CA91895: to FM-B.). All authors are independent of the funding agencies.
, http://www.100md.com
Competing interests: None declared.
Ethical approval: Not required.
References
Satterlund MJ, McCaul KD, Sandgren AK. Information gathering over time by breast cancer patients. J Med Internet Res 2003;5: e15.
Fox S, Rainie L. The online health care revolution: how the web helps Americans take better care of themselves. Washington DC: Pew Internet and American Life Project: Online, 2000: 3-7.
, 百拇医药
Hope and the media in advanced cancer patients. American Society of Clinical Oncology 36th annual meeting; New Orleans, 2000.
Eisenberg DM, Kessler RC, Rompay MIV, Kaptchuk TJ, Wilkey SA, Appel S, et al. Perceptions about complementary therapies relative to conventional therapies among adults who use both: results from a national survey. Ann Intern Med 2001;135: 344-51.
Biermann JS, Golladay GJ, Greenfield ML, Baker LH. Evaluation of cancer information on the Internet [see comments]. Cancer 1999;86: 381-90.
, 百拇医药
Jadad AR, Gagliardi A. Rating health information on the internet: navigating to knowledge or to Babel JAMA 1998;279: 611-4.
Price SL, Hersh WR. Filtering web pages for quality indicators: an empirical approach to finding high quality consumer health information on the world wide web. Proc AMIA Symp 1999: 911-5.
Weinberg N, Schmale J, Uken J, Wessel K. Online help: cancer patients participate in a computer-mediated support group. Health Soc Work 1996;21: 24-9.
, 百拇医药
Hoch DB, Norris D, Lester LE, Marcus AD. Information exchange in an epilepsy forum on the world wide web. Seizure 1999;8: 30-4.
Mursch K, Behnke-Mursch J. Internet-based interaction among brain tumour patients. Analysis of a medical mailing list. Zentralbl Neurochir 2003;64: 71-5.
Ferguson T. Heatlh online: how to find health information, support groups and self help communities in cyberspace. Reading, MA: Addison-Wesley, 1996.
, 百拇医药
Feenberg AL, Licht JM, Kane KP, Moran K, Smith RA. The online patient meeting. J Neurol Sci 1996;139(Suppl): 129-31.
Culver JD, Gerr F, Frumkin H. Medical information on the internet: a study of an electronic bulletin board. J Gen Intern Med 1997;12: 466-70.
Till JE. Evaluation of support groups for women with breast cancer: importance of the navigator role. Health Qual Life Outcomes 2003;1: 16., http://www.100md.com(Adol Esquivel, Funda Meri)