Evaluation based on scientific publishing: Evaluating researchers and research groups

Evaluating researchers and research groups

Evaluation of researchers or research groups must follow the principles of responsible evaluation/metrics. Recommendations in Finland:

The most used metrics are 1) production of publications (sum of publications) per year 2) number of citations. Scholarly citations can be retrieved from Web of Science - and Scopus -databases. At the same it is possible to get the total number and share (%) of those publications that are indexed in above mentioned databases.  Citations and H-index can be calculated for all publications or for the last 3-5 years as well as with or without self-citations. In addition field normalized indexes are used d.g. Field Weighted Citation Index (FWCI) which is available in SciVal-database.

Also the openness of articles  is looked at. Read more

Altmetrics explores the visibility and attention publications have received in social media and scholarly networking tools.

The esteem of an individual researcher can be also evaluated on the basis of other factors: the networking of the researcher, being a member of an advisory board, acting as an expert in peer reviewing, presentations and being a keynote speaker, and awards and recognition etc. Often the information is only available from the researcher's or research group's own website.

Evaluating research groups, e.g. in biosciences and medicine, is important because almost all research is conducted in groups. In addition, the groups are evaluated with expert panels; for example, when they are applying for funding, for membership in separate programs in university consortia, research organisations and graduate schools, and in applications for the Centres of Excellence of the Academy of Finland.

Quality of publications

In Finland the most important tool for evaluation of publication channels is Publication Forum. The classification system was originally meant for the evaluation of the average quality of a large number of publications produced by universities. The classification is not meant for  the evaluation of the quality of a smaller number of publications produced by the units of universities or other research organisations or individual publications – articles or monographs – nor for the evaluation or comparison of individual researchers. Read more 

Globally the most known tool for evaluation of journal is the journal Impact Factor, IF which also should not be used for the evaluation of individual researhers.

Read more about the journal evaluation.

Research productivity and publications types

The most reliable way for evaluation the production of publications is to use researcher´s or research group´s publication list.

When searching researcher´s publications from international databases it is recommended to use the author-search.

In finnish CRIS-systems, like Oulun yliopisto tutkii,  publications are classified according to the publication type classification by the Ministry of Culture and Education

It is important to remember that the number of publications does not correlate with the quality of publications.

Databases containing citation information

The most important multidisciplinary databases containing citation information are Web of Sciece (WoS) by  Clarivate and Scopus by Elsevier.  Citations can also be retrieved from Google Scholar (GS) keeping in mind the limitations of the database. GS contains a lot of non-scientific citations. The number of citations can vary a lot between Web of Science, Scopus and especially GS.

In addition, there are some field specific databases which contain reference information such as Chemical Abstracts (SciFinder), CiteSeerX and MathScinet.

Read more

Research collaboration

The research collaboration can be evaluated looking the share of international, national, in-side organization or single author publications. Intresting point is also how much research is done with companies or industry.

Top lists

Top lists are often based on the amount of citations the researcher's publications have received. Only the most productive and respected scholars are to be found in the top lists, therefore only rare researchers can be evaluated on the basis of these lists. 

Examples of  top lists on researchers::

  • ScienceWatch.com: ScienceWatch.com: not only contains the most cited researchers top lists, but also other data that can aid the evaluation, such as information about researchers, the publications, the organisations and even the countries.
  • HighlyCited.com is not a traditional top list, but it does contain only the most cited researchers in different disciplines.
  • CiteSeerX: CiteSeerX contains rankings of the most cited researchers in computer and information sciences.