Evaluation based on scientific publishing: Citation metrics of journals

Citation metrics of journals

Impact Factor calculations are based on citation analysis and this is seen to be a more objective measure than expert evaluation. The evaluation of the quality of a journal on the basis of how many citations the articles that are published in that journal have had, is based on an assumption that the those articles that are often cited contain significant knowledge for their discipline; they are therefore highly influential. A journal's Impact Factor which is calculated on the basis of the citation statistics can only be calculated years after publication of the journal, because those citations only appear in articles which are published later. Journal specific Impact Factor indicators are always some sort of average of the Impact Factors of the articles published in them.

Presently there are many different types of indicators of impact, and they are calculated in different ways. These indicators can be split up in to two groups, based on how they measure impact. Indicators based on the number of citations like the Impact Factor (IF), measure the popularity, while indicators that include different weighting in the citation analysis can measure the prestige of the journal. All indicators are calculated from a specific group of journals, and the citations from those journals are counted; for example, journals that are part of the Thomson Reuters' database. This makes it difficult to compare different indicators; they differ from each other not only in the method of calculation, but also the calculations are based on journal and citation material from different databases. Some indicators are based on simple and easy to understand calculation formulas, but on the other hand you do meet more elaborate algorithms such the PageRank algorithm.

It is good to remember that presently widely used indicators are only one choice among many possible alternatives, and that indicators are constantly under criticism. New indicators are being developed all the time, and existing indicators are evolving. All indicators of impact are calculated by dividing the amount of times articles have been cited from a certain journal by the amount of articles published in the journal. Indicators vary according to how long the chosen period is for calculating the number of citations, and what citations are included. In addition, the calculation on the amount of articles fluctuates; some indicators calculate all different types of articles that have been published in a journal, and some indicators only take in to consideration a certain type of article. Different indicators also vary from each other according to whether all citations are viewed of as being equally valuable or not.

The calculated values for the indicators from a particular database are based on the amount of cited material in that database. As a result, the score for the same indicator can differ significantly between different databases. Indicators measure impact through citations, but impact does not always mean quality. One can also only evaluate a small number of academic publications with such indicators, because the majority of academic articles are not cited in other articles.

Journal Immediacy index

Thomson Reuters also publishes in its Journal Citation Reports Journal an index called Immediacy Index, which is the average number of times an article is cited in the year it is published. The Immediacy Index is particularly useful for comparing journals specializing in cutting-edge research, but it can also be used to evaluate newly added journals, which do not currently have Impact Factor values. Read more about Immediacy Index.

The impact of Open Access Journals

In 2004, the Thomson database had 239 Open Access journals. This group included many different types of journals: new journals that have always been available for all, and old journals that have changed from being traditional journals to which you had to subscribe to having either partly or entirely free electronic access. Medicine and the Biosciences have the most OA journals, but Physics, Engineering and Mathematics OA journals are among the leading journals of their subject area.
Most OA journals belonged to the lower scoring half of the evaluated journal ranking lists in their subject area. OA journals, however, are situated higher in the ranks in the Immediacy Index scores than when evaluated with IF scores. The high Immediacy Index score tell us that the article is cited soon after it had been published, and that its content is topical at the time of publication. All in all, one cannot presume that the availability affects the quality measures of a journal, rather the impact of the journal is determined by its content.