Where Should Family Medicine Papers be Published—Following the Impact Factor
http://www.100md.com
《家庭医学美国版医学期刊》
the Department of Family Medicine, Sial Family Medicine and Primary Care Research Center, Faculty of Health Sciences, Ben-Gurion University of the Negev, Beer-Sheva, Israel
Abstract
Academic institutions weigh the research contribution of family physicians and take this factor into account when determining eligibility for the candidates’ promotion. Among other parameters, these institutions consider the journals in which family physicians publish. In this respect, the impact factor (IF) has gained a foothold as one of the most accepted means to measure this contribution. The IF may be a measure of the main importance of a scientific journal. IF has a huge, but controversial, influence on the perception and evaluation of published scientific research. It is important for family physicians to understand and be aware of the importance of the IF and the way it is calculated. The IF is one consideration in the decision-making process of a researcher as to where to publish because the IF of most family medicine journals is less than 2.0. Thus publication in these journals might not yield the proper "score" for academic promotion in many institutions. On the other hand, publication in journals with higher IF that are not necessarily widely read by primary care physicians could result in a small impact of their findings on direct patient care.
Research in family medicine is an essential component of the work of academic family doctors and is an important way to better understand community medicine and to provide better care for patients. Publications of this research form the basis for recommendations regarding basic clinical decisions, epidemiology of common problems, understanding etiologies, appropriate use of diagnostic tools, health policy, therapy, and guidelines in the family medicine and primary care setting. In recent years, there has been an increase in the number of family physicians who are actively involved in research and publish scientific papers. Unlike other medical specializations in which most investigators publish papers in a specific field that is related to their specialization, family physicians, being generalists, publish in family medicine journals and in other journals covering different medical specialties.
In a cross-sectional comparison of family medicine originated papers from the United States in the years 1979 and 1989, it was found that only one half of the articles were published in family medicine journals and that this proportion did not change during this period.1 Family physicians submit papers to non-family medicine journals often because they perceive that non-family medicine journals have more prestige and that institutional promotion and tenure committees give less weight to publications in family medicine journals.2
Stratifying candidates objectively on the merit of publication portfolios is an onerous and difficult task. Institutional committees are under increasing pressure to rank applicants based on previous achievements for appointments/promotions, funding, and awards, and must do so within unforgiving time constraints. The journal impact factor (IF) has been loosely adopted in many circles for assessing article "quality", circumventing detailed review of individual articles. The premise supporting such practice often hinges on assuming that high IF journals are harder to publish in.
Indeed, academic institutions do weigh the academic contribution of family physicians and take this factor into account when determining eligibility for academic promotion. Among other parameters these institutions consider the journals in which family physicians publish. In this respect the IF has gained a foothold as the accepted mean to measure this contribution.
What is the impact factor
The IF is a trial to measure of the importance of a scientific journal. The Institute of Scientific Information (ISI) tracks journals, calculates their IF and publishes it annually in its Journal Citation Report. The IF has a huge, but controversial, influence on the perception and evaluation of published scientific research.3 It is important for family physicians to understand and be aware of the presumed importance of the IF and the way it is calculated.
In 1955 Eugene Garfield established the ISI, which developed a bibliographic method for the advancement and evaluation of scientific literature—the Citation Index for Science (SCI).4 It mainly consisted of a complete alphabetic listing of all periodicals covered and their representative codes. These codes described the bibliography category (eg, editorial, original research, review), whereas a different set of data were assigned to articles referring to an article in question.5 This method helped to codify scientific publications and to classify them by type of publication such as original article, review paper, editorial, letter to the editor. The value of the IF is obtained by calculating the number of times papers published in a specific journal over the previous 2 years were cited in other publications in the current year. This value is then divided by the number of publications that appeared in that journal over the same period of time. Thus, the IF of journal A in year Y would be determined by the formula5:
Per definition, the only papers considered in ISI are original research papers, technical papers, and reviews. Editorials, letters to the editor, news items, abstracts and proceedings, are not included in the calculation.5
The following example should help to clarify IF calculation. The New England Journal of Medicine published 366 "citable" articles in 2003 and 378 in 2002. The articles from 2003 were cited 14,147 times in 2004, and the articles from 2002 were cited 14,549 times.5 Thus, the IF for 2004 would be:
The SCI covers less than 25% of all the journals that are listed as peer-reviewed journals in the world.6 English language journals are clearly preferred in the IF index and local or regional journals, printed in a language other than English, are clearly underrepresented in IF.5 Additional factors such as the area of research may also influence IF. For example, basic research in medical articles are published 3 to 5 times more than clinical articles.5,7
Today, publication in the electronic media is becoming more and more common, and this does affect the IF. Articles that are published in full in the electronic media are much more likely to be cited than articles that appear only as abstracts in the same media.9,10
The Euro-factor (EF) and Cited Half-Life IF (CHAL-IF) are other trials to evaluate the academic weight of a journal. The EF database was proposed as an alternative to meet the citation measurement demand of the European scientific community, targeting the language bias and perceived United States centricity. However, the formula does not further the understanding of how a European-specific ranking is achieved, and the EF coefficient was arbitrarily set at a value of 10, leading to wide differences between the EF and IF for the same journals.10
The CHAL-IF is calculated by replacing the 2-year citation window with the journal’s CHAL in the IF computation formula. Expansion of the time frame for calculation results in a better stability of the average IF by the CHAL method than that calculated by the standard ISI method.11 Rousseau renamed the CHAL-IF to median IF (MIF) and furthered developed it to the percentile IF (pIF), whereby the MIF is the pIF set at 50%.12
Other alternative measures have included the journal to field impact score (JFIS) and the disciplinary IF (DIF).5 These modifications of the IF are meant as a complementary assessment tool. Nonetheless, the IF remains the most accepted method.
Discussion
The IF is not a perfect tool for the evaluation of journals. It has many shortcomings. It does not evaluate individual articles13 and cannot, as a single index, assess the quality of work of specific authors. Yet, in an increasing "publish or perish" clinical and academic environment, clinicians involved in research should be acquainted with this assessment tool, which still serves to evaluate individuals for academic promotion in many institutions.14
The leading family medicine journals have IFs lower than 2.0. Academic institutions that consider IFs for faculty promotion may not look favorably on those candidates who enjoy publication in these journals. This may lead many family physicians to publish their best work in other journals relating to other specialties, such as pediatrics (IF range 0.374 to 3.903), pain and palliative care journals (IF range 0.837 to 4.061), dermatology (IF range 0.708 to 3.187), gynecology (IF range 0.474 to 3.512), and psychiatry (IF range 0.581 to 11.207) (all IF data from 2004). Journals in medical education, health services organization are not different in their IF range than family medicine. The IF in general journals is promising and reaches up to 38.57. However, to advise researchers to seek high IF journals is not always beneficiary. Basic science journals tend to have higher IF because they are read (and cited) by other researchers. Primary care and clinical practice journals are widely read by clinicians but these practitioners are not as likely to be actively publishing and citing others. So, a clinical journal with a wide readership can have a lot of impact as clinicians incorporate new knowledge into their patient care, but this is not captured by the IF. This is a huge bias of the IF that is particularly relevant to our readership. Saha et al15 found the evaluation of journals by both clinicians and health services researchers was associated with the IFs. However, among the practitioners group the correlation was much lower (0.62) than the researchers group (0.83). This further emphasizes the biases relating to IF as a measure of importance to clinicians.
Studies examining relevant regional issues are often published in the local language and in the local journals, where the message has regional importance. Such niche journals (those with selective readership) will always be disadvantaged in their IF as their readership (and hence the number of potential citations) is relatively small. Maybe one way of improving the IF is to formulate an index that includes an annual number of subscribers to a journal and/or the number entering a journal website.
Other quality criteria authors might want to consider when they choose a journal are the speed of editorial processing (time from submittal to first decision, time from acceptance to publication), the reputation of the editors and editorial board, and the accessibility of the journals (open access or not, for example).16,17
There is a conflict between what is good for patient care (local publication) and what is good for a physician’s academic career (publishing in "international" high IF, general rather than specialty journals). National, regional, and local institutions need to face this conflict and rethink their research evaluation policies for health researchers, because although these policies have succeeded in increasing the output of articles published in higher IF journals, it is not at all clear that they have aided the efficient dissemination of high-quality information to practitioners who could be using this information to take better care of their patients.
Notes
This article was externally peer-reviewed.
Conflict of interest: none declared.
References
Ingram TG. A cross-sectional analysis of family medicine publications in the indexed medical literature. Fam Med 1992; 24: 303–6.
Weiss BD. Publications by family physicians in non-family medicine journals. Fam Pract Res J 1990; 10: 117–22.
Wikepedia the free encyclopedia. Available from: http://en.wikepedia.org/wiki/Impact factor "and references therein" Accessed 5 September 2006.
Garfield E. Citation indexes for science; a new dimension in documentation through association of ideas. Science 1955; 122: 108–11.
Dong P, Loh M, Mondry A. The "impact factor" revisited. Biomedical Digital Libraries 2005; 2: 7.
Moed HF, Burger WJM, Frankfort JG, Van Raah AFJ. On the measurement of research performance: the use of bibliometric indicators. Leiden, Science Studies Unit, Lisbon, Institute University of Leiden, 1987.
Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ 1997; 314: 498–502.
BioMed Central—the open access publisher. Available from: http://www.biomedcentral.com
Murali NS, Murali HR, Auethavekiat P, et al. Impact of FUTON and NAA bias on visibility of research. Mayo Clin Proc 2004; 79: 1001–6.
Hafbauer R, Frass M, Gmeiner B, Kaye AD. Euro-Factor—the new European scientific currency. Vienna: VICER Publishing: 2002.
Sombatsompop N, Markpin T, Premkamolnetr N. A modified method for calculating the impact factors of journals in ISI journal citation reports: Polymer Science Category in 1997–2001. Scientometrics 2004; 60: 217–35.
Rousseau R. Median and percentile impact factor: a set of new indicators. Scientometrics 2005; 63: 431–41.
Opthof T, Coronel R, Piper HM. Impact factors: no totum pro parte by skewness of citation. Cardiovasc Res 2004; 61: 201–3.
Cartwright VA, McGhee CN. Ophthalmology and vision science research. Part 1: Understanding and using journal impact factors and citation indices. J Cataract Refract Surg 2005; 31: 1999–2007.
Saha S, Saint S, Christakis DA. Impact factor: a valid measure of journal quality J Med Libr Assoc 2003; 91: 42–6.
Jefferson T, Wager E, Davidoff F. Measuring the quality of editorial peer review. JAMA 2002; 287: 2786–90.
Jefferson T, Alderson P, Wager E, Davidoff F. Effects of editorial peer review: a systemic review. JAMA 2002; 287: 2784–6.(Roni Peleg, MD and Pesach Shvartzman, MD)
Abstract
Academic institutions weigh the research contribution of family physicians and take this factor into account when determining eligibility for the candidates’ promotion. Among other parameters, these institutions consider the journals in which family physicians publish. In this respect, the impact factor (IF) has gained a foothold as one of the most accepted means to measure this contribution. The IF may be a measure of the main importance of a scientific journal. IF has a huge, but controversial, influence on the perception and evaluation of published scientific research. It is important for family physicians to understand and be aware of the importance of the IF and the way it is calculated. The IF is one consideration in the decision-making process of a researcher as to where to publish because the IF of most family medicine journals is less than 2.0. Thus publication in these journals might not yield the proper "score" for academic promotion in many institutions. On the other hand, publication in journals with higher IF that are not necessarily widely read by primary care physicians could result in a small impact of their findings on direct patient care.
Research in family medicine is an essential component of the work of academic family doctors and is an important way to better understand community medicine and to provide better care for patients. Publications of this research form the basis for recommendations regarding basic clinical decisions, epidemiology of common problems, understanding etiologies, appropriate use of diagnostic tools, health policy, therapy, and guidelines in the family medicine and primary care setting. In recent years, there has been an increase in the number of family physicians who are actively involved in research and publish scientific papers. Unlike other medical specializations in which most investigators publish papers in a specific field that is related to their specialization, family physicians, being generalists, publish in family medicine journals and in other journals covering different medical specialties.
In a cross-sectional comparison of family medicine originated papers from the United States in the years 1979 and 1989, it was found that only one half of the articles were published in family medicine journals and that this proportion did not change during this period.1 Family physicians submit papers to non-family medicine journals often because they perceive that non-family medicine journals have more prestige and that institutional promotion and tenure committees give less weight to publications in family medicine journals.2
Stratifying candidates objectively on the merit of publication portfolios is an onerous and difficult task. Institutional committees are under increasing pressure to rank applicants based on previous achievements for appointments/promotions, funding, and awards, and must do so within unforgiving time constraints. The journal impact factor (IF) has been loosely adopted in many circles for assessing article "quality", circumventing detailed review of individual articles. The premise supporting such practice often hinges on assuming that high IF journals are harder to publish in.
Indeed, academic institutions do weigh the academic contribution of family physicians and take this factor into account when determining eligibility for academic promotion. Among other parameters these institutions consider the journals in which family physicians publish. In this respect the IF has gained a foothold as the accepted mean to measure this contribution.
What is the impact factor
The IF is a trial to measure of the importance of a scientific journal. The Institute of Scientific Information (ISI) tracks journals, calculates their IF and publishes it annually in its Journal Citation Report. The IF has a huge, but controversial, influence on the perception and evaluation of published scientific research.3 It is important for family physicians to understand and be aware of the presumed importance of the IF and the way it is calculated.
In 1955 Eugene Garfield established the ISI, which developed a bibliographic method for the advancement and evaluation of scientific literature—the Citation Index for Science (SCI).4 It mainly consisted of a complete alphabetic listing of all periodicals covered and their representative codes. These codes described the bibliography category (eg, editorial, original research, review), whereas a different set of data were assigned to articles referring to an article in question.5 This method helped to codify scientific publications and to classify them by type of publication such as original article, review paper, editorial, letter to the editor. The value of the IF is obtained by calculating the number of times papers published in a specific journal over the previous 2 years were cited in other publications in the current year. This value is then divided by the number of publications that appeared in that journal over the same period of time. Thus, the IF of journal A in year Y would be determined by the formula5:
Per definition, the only papers considered in ISI are original research papers, technical papers, and reviews. Editorials, letters to the editor, news items, abstracts and proceedings, are not included in the calculation.5
The following example should help to clarify IF calculation. The New England Journal of Medicine published 366 "citable" articles in 2003 and 378 in 2002. The articles from 2003 were cited 14,147 times in 2004, and the articles from 2002 were cited 14,549 times.5 Thus, the IF for 2004 would be:
The SCI covers less than 25% of all the journals that are listed as peer-reviewed journals in the world.6 English language journals are clearly preferred in the IF index and local or regional journals, printed in a language other than English, are clearly underrepresented in IF.5 Additional factors such as the area of research may also influence IF. For example, basic research in medical articles are published 3 to 5 times more than clinical articles.5,7
Today, publication in the electronic media is becoming more and more common, and this does affect the IF. Articles that are published in full in the electronic media are much more likely to be cited than articles that appear only as abstracts in the same media.9,10
The Euro-factor (EF) and Cited Half-Life IF (CHAL-IF) are other trials to evaluate the academic weight of a journal. The EF database was proposed as an alternative to meet the citation measurement demand of the European scientific community, targeting the language bias and perceived United States centricity. However, the formula does not further the understanding of how a European-specific ranking is achieved, and the EF coefficient was arbitrarily set at a value of 10, leading to wide differences between the EF and IF for the same journals.10
The CHAL-IF is calculated by replacing the 2-year citation window with the journal’s CHAL in the IF computation formula. Expansion of the time frame for calculation results in a better stability of the average IF by the CHAL method than that calculated by the standard ISI method.11 Rousseau renamed the CHAL-IF to median IF (MIF) and furthered developed it to the percentile IF (pIF), whereby the MIF is the pIF set at 50%.12
Other alternative measures have included the journal to field impact score (JFIS) and the disciplinary IF (DIF).5 These modifications of the IF are meant as a complementary assessment tool. Nonetheless, the IF remains the most accepted method.
Discussion
The IF is not a perfect tool for the evaluation of journals. It has many shortcomings. It does not evaluate individual articles13 and cannot, as a single index, assess the quality of work of specific authors. Yet, in an increasing "publish or perish" clinical and academic environment, clinicians involved in research should be acquainted with this assessment tool, which still serves to evaluate individuals for academic promotion in many institutions.14
The leading family medicine journals have IFs lower than 2.0. Academic institutions that consider IFs for faculty promotion may not look favorably on those candidates who enjoy publication in these journals. This may lead many family physicians to publish their best work in other journals relating to other specialties, such as pediatrics (IF range 0.374 to 3.903), pain and palliative care journals (IF range 0.837 to 4.061), dermatology (IF range 0.708 to 3.187), gynecology (IF range 0.474 to 3.512), and psychiatry (IF range 0.581 to 11.207) (all IF data from 2004). Journals in medical education, health services organization are not different in their IF range than family medicine. The IF in general journals is promising and reaches up to 38.57. However, to advise researchers to seek high IF journals is not always beneficiary. Basic science journals tend to have higher IF because they are read (and cited) by other researchers. Primary care and clinical practice journals are widely read by clinicians but these practitioners are not as likely to be actively publishing and citing others. So, a clinical journal with a wide readership can have a lot of impact as clinicians incorporate new knowledge into their patient care, but this is not captured by the IF. This is a huge bias of the IF that is particularly relevant to our readership. Saha et al15 found the evaluation of journals by both clinicians and health services researchers was associated with the IFs. However, among the practitioners group the correlation was much lower (0.62) than the researchers group (0.83). This further emphasizes the biases relating to IF as a measure of importance to clinicians.
Studies examining relevant regional issues are often published in the local language and in the local journals, where the message has regional importance. Such niche journals (those with selective readership) will always be disadvantaged in their IF as their readership (and hence the number of potential citations) is relatively small. Maybe one way of improving the IF is to formulate an index that includes an annual number of subscribers to a journal and/or the number entering a journal website.
Other quality criteria authors might want to consider when they choose a journal are the speed of editorial processing (time from submittal to first decision, time from acceptance to publication), the reputation of the editors and editorial board, and the accessibility of the journals (open access or not, for example).16,17
There is a conflict between what is good for patient care (local publication) and what is good for a physician’s academic career (publishing in "international" high IF, general rather than specialty journals). National, regional, and local institutions need to face this conflict and rethink their research evaluation policies for health researchers, because although these policies have succeeded in increasing the output of articles published in higher IF journals, it is not at all clear that they have aided the efficient dissemination of high-quality information to practitioners who could be using this information to take better care of their patients.
Notes
This article was externally peer-reviewed.
Conflict of interest: none declared.
References
Ingram TG. A cross-sectional analysis of family medicine publications in the indexed medical literature. Fam Med 1992; 24: 303–6.
Weiss BD. Publications by family physicians in non-family medicine journals. Fam Pract Res J 1990; 10: 117–22.
Wikepedia the free encyclopedia. Available from: http://en.wikepedia.org/wiki/Impact factor "and references therein" Accessed 5 September 2006.
Garfield E. Citation indexes for science; a new dimension in documentation through association of ideas. Science 1955; 122: 108–11.
Dong P, Loh M, Mondry A. The "impact factor" revisited. Biomedical Digital Libraries 2005; 2: 7.
Moed HF, Burger WJM, Frankfort JG, Van Raah AFJ. On the measurement of research performance: the use of bibliometric indicators. Leiden, Science Studies Unit, Lisbon, Institute University of Leiden, 1987.
Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ 1997; 314: 498–502.
BioMed Central—the open access publisher. Available from: http://www.biomedcentral.com
Murali NS, Murali HR, Auethavekiat P, et al. Impact of FUTON and NAA bias on visibility of research. Mayo Clin Proc 2004; 79: 1001–6.
Hafbauer R, Frass M, Gmeiner B, Kaye AD. Euro-Factor—the new European scientific currency. Vienna: VICER Publishing: 2002.
Sombatsompop N, Markpin T, Premkamolnetr N. A modified method for calculating the impact factors of journals in ISI journal citation reports: Polymer Science Category in 1997–2001. Scientometrics 2004; 60: 217–35.
Rousseau R. Median and percentile impact factor: a set of new indicators. Scientometrics 2005; 63: 431–41.
Opthof T, Coronel R, Piper HM. Impact factors: no totum pro parte by skewness of citation. Cardiovasc Res 2004; 61: 201–3.
Cartwright VA, McGhee CN. Ophthalmology and vision science research. Part 1: Understanding and using journal impact factors and citation indices. J Cataract Refract Surg 2005; 31: 1999–2007.
Saha S, Saint S, Christakis DA. Impact factor: a valid measure of journal quality J Med Libr Assoc 2003; 91: 42–6.
Jefferson T, Wager E, Davidoff F. Measuring the quality of editorial peer review. JAMA 2002; 287: 2786–90.
Jefferson T, Alderson P, Wager E, Davidoff F. Effects of editorial peer review: a systemic review. JAMA 2002; 287: 2784–6.(Roni Peleg, MD and Pesach Shvartzman, MD)