Evaluation based on scientific publishing: Journal Impact Factor, IF

Journal Impact Factor, IF

The Journal Impact Factor can be used to evaluate or compare a journal's relative importance to other journals in the same field. It shows an average citation rate per article for the past two years. Thomson Reuters publishes annually Journal Impact Factors for journals included in Web of Science database in its Journal Citation Reports.

Calculation of IF

The IF for a journal (j) is defined as the number of times articles published in the two years prior to the JCR edition year have been cited in the JCR edition year (C) divided by the number of articles published in those two years (P).

JIFj = C ⁄ P

A diagram of Impact Factor calculation.

It should be noted that only research papers (articles) and review articles (citable articles) are considered in the calculation of the denominator. Letters, editorials etc are not taken into account. However, the numerator reflects citations to all document types including self-citations. Normally the non-citable articles are seldom cited, and thus do not considerably affect the IF of a journal. JCR also provides journal IFs without journal self-cites.

The two-year period used in calculating the IF is a compromise. If it were based on one year's articles only, it would give even greater weight to rapidly changing fields. If it were based on longer periods of citations and sources, the measure would be less current.

The calculation of IF for Cell (2007).
Image source: Journal Citation Reports (Thomson Reuters)<http://www.isiknowledge.com/> 2.4.2009

Factors influencing the IF statistics

  • The IF is relatively independent of the size of a journal (number of articles published by the journal), because it is a relative metric. However, journals with large number of articles published usually have high IF values, because they are more likely to contain highly cited articles than journals with small numbers of articles published. Sudden changes in a journal's size can affect the Impact Factor. If the number of articles published is suddenly decreased, it can cause a temporaral rise of the IF of a journal.
  • The size of the scientific community that a journal serves does not significantly affect the journal's IF. While more authors produce more citations, these must be shared by a larger number of cited articles.
  • Because the IF is determined by citation frequency, it is a valid measure for journal impact for scientists who publish and generate citations. However, it excludes the frequency of journal use, which may be significant in case of some medical journals.
  • The type of journal, rather than the quality alone, may influence the Impact Factor. Journals publishing a large number of review articles usually have high Impact Factors, because review articles usually have a higher average rate of citation than original research papers. If a journal publishes a large number of reviews one year, there may be a temporary increase in the number of citations received. Additionally in some fields methodological papers are also more cited than original research papers.
  • The distribution of citations of articles within a journal is skewed, a relatively small number of articles within a single journal receive the majority of all citations. The well-known 80/20 rule applies that 20% of articles may account for 80% of the citations. And a large percentage of published articles are never cited.
  • In the first year after a journal title change, the new title is listed without an Impact Factor because the article count for two preceding years used in the Impact Factor calculations is zero. This also concerns new titles included in the database.

Fluctuations of the IF

The Impact Factors of journals fluctuate from year to year, and part of this variation is random. The relatively small number of papers contributing to the Impact Factor means that there is a large random component in the variation. As there are many more journals with a low Impact Factor than journals with a high one, rankings for the low impact ones are less stable than for the high impact ones. For high impact journals, noise and fluctuations have only a small influence on the impact, and do not lead to any change in ranking. However, journals are often compared for a fixed year, without taking into account the higher variation for small journals.

Limitations and biases of the Impact Factor

  • All citations are weighted equally regardless of the prestige of the citing journal.
  • Differences in citation patterns among disciplines are not considered, which makes it impossible to compare journals across disciplines. The different disciplinary citation patterns are correlated with the size of the pool of citable literature, different citation traditions, the length of articles, the number of references in articles, the average amount of articles per author and variations in the use of indirect citations, which means citing only one specifically referenced publication, and adding a phrase such as "and references therein".
  • The context of citations is not considered. Articles are cited for various reasons, and often a citation does not reflect the scientific merit of the cited work. Negational citing is especially related to disciplines with a critical discourse. Thus a high citation rate of an article may not always be associated with high quality.
  • Only articles that are cited within two years after publication contribute to the impact factor although many important papers achieve their maximal scientific impact outside of this time-frame. The arbitrary selection of a two-year reference period has been the subject of much debate.
  • Citations made in journals not recorded in the Web of Science database do not contribute to impact factor calculations.
  • Citations are gathered from journal articles only, and not from books, book chapters, or conference proceedings.
  • The denominator of the Impact Factor calculation is made up by citable articles. The non-citable items, including letters, news stories, abstracts, book reviews, and editorials, are not included in the denominator of the impact factor equation but may be included in the numerator if they are, although infrequently, cited.
  • The sample of journals included in JCR is limited, and represents only a small fraction of all journals. In addition, the journal selection process is somewhat ambiguous. The content of the JCR is not static, but the journal evaluation and selection process is an ongoing process.
  • JCR has a preference for the English language, and is dominated by North American publications. Journals that publish articles in languages other than English will likely receive fewer citations because a large portion of the scientific community cannot read them.
  • Self-citations are included in total cites of a journal. A high self-cited rate indicates a journal's low visibility, and a high self-citing rate is an indicator of the isolation of the field covered by the journal. Thomson Reuters takes into consideration the self-citation rates of journals it covers, and an unusually high rate of self-citations can lead to exclusion of the journal. The total citation counts also include citations of authors to their own work but several citations to same article in one article are counted only once. JCR also provides journal IFs without journal self-cites.
  • Minor errors in the preparation of article reference list information (basic typographical mistakes, inconsistent spelling of surnames and use of initials, and so on) may also inadvertently bias citation identification and recording. It has been suggested that this type of mistake occurs in up to 7% of citations.
  • Comparisons of two journals in different disciplines using Impact Factor are not possible. The Impact Factor varies considerably among disciplines due to different citation and referencing tendencies of each discipline. In addition, if in some disciplines many citations occur outside the two-year window, Impact Factors for journals will be low.
  • The Impact Factor of a journal cannot be considered to represent the citation rate of an individual article and does not permit assessment of the quality of an individual article or author. In fact, there is virtually no correlation between the frequency of citation of an individual article and the Impact Factor of the publishing journal. The misuse of Impact Factor has, in recent years, widened to include evaluation of the quality of individual papers and even individual authors and research groups.
  • Many authors may be tempted, or feel pressured, to select the highest Impact Factor-rated journals likely to accept their article for publication while rejecting journals whose target audience may in fact be more suitable and receptive to the publication itself.
  • Impact Factor can distort publication polices of journals, if they focus on improving their Impact Factors for example by publishing more reviews. An item is classified as a review by Thomson Reuters if it meets any of the following criteria: it cites more than 100 references, it appears in a review publication or a review section of a journal, the word review or overview appears in its title, or the abstract states that it is a review or survey.