Evaluation based on scientific publishing: Evaluating disciplines and research fields

Katso myƶs

- Maiden arviointi
- Journal Citation Reports

Evaluation

When evaluating scientific performance, the evaluation criteria are publishing productivity, and the quality of publications, mainly on the basis of citation analysis. When evaluating different disciplines, they need to be looked at separately, as they differ in terms of publishing practices, referencing practices and also the speed of obsolescence of scientific knowledge and literature.

Measuring the publishing productivity of different disciplines is complicated. The amount of work required in research in different disciplines varies because of the different nature of the research, and different publishing types require a different amount of work. Hence, when measuring publishing productivity, different forms of publishing need to be weighted differently. The weighting should take into account that different forms of publishing command differing degrees of respect from discipline to discipline.One also needs to consider the different resources the units under evaluation had, and the differences between disciplines in terms of funding and the time taken for the research to be produced.

The varying co-authoring norms and practices will also affect the publishing productivity of the discipline. Joint publications can be calculated in several different ways when looking at the publishing activities of a department. One way is to add to the publications of a whole department all publications that have at least one department member as a co-author. Joint publications can also be divided in different ways, for example on the basis of the number of writers that have been involved in them.

Evaluation based on citation information is not best suited for those disciplines that concentrate on forms of publication other than international journal articles. The databases with the citation information concentrate on international peer reviewed journals, and do not cover systematically other forms of publishing. Their journal collection is also centred around journals from the United States and English language journals, and their coverage of Social Science and Arts is poor, even with regard to international journals. Using citation data as the tool of evaluation is only applicable to those disciplines in which international journal articles are the main publishing forums.

In some technical and science fields, the most highly regarded publications are papers presented at the best conferences. The quality of conferences is primarily evaluated on the basis of the acceptance and rejection rates of submitted papers.

Different evaluation criteria should be developed for those disciplines where international journal articles are not the predominant publishing form. Expert panels are used e.g. in the European Reference Index for the Humanities and the Social Sciences (ERIH PLUS), and in Finnish Publication Forum (Julkaisufoorumi).

The Academy of Finland conducts evaluations of disciplines and individual fields of research. The purpose and objective of evaluation is to gain an expert and independent view of the state of the discipline in question, of the quality of its research and its development needs. The researchers and experts appointed to the evaluation panel come predominantly from outside Finland.