Evaluation based on scientific publishing: Evaluating countries

Katso myƶs

- Arvioinnin indikaattoreita
- Tieteenalojen arviointi
- Artikkelien arviointi
- Tutkijoiden ja tutkimusryhmien arviointi
- SCImago Journal and Country Rank
- ScienceWatch

Evaluating countries

In order to assess the quality and productivity according to country, the most commonly used measures are the numbers of publications and impact indicators that are calculated according to citations, along with expert evaluations in discipline specific assessments.

Interpreting bibliometric indicators on a macro level is not easy when evaluating whole countries, and therefore this requires particular caution and critical thinking. Country specific assessments are usually based on several indicators, but the results often contradict each other. When comparing different countries, the rankings can vary vastly according to which indicator is being examined. This does open up the possibility of choosing the most 'convenient' indicator for some specific purpose.

Even the most simple of indicators such as how many publications a country has produced can be calculated in so many different ways. There are at least three ways to calculate the collaborative publications researchers have produced across national borders. The publications can be attributed to the collaborating countries according to how many researchers from each country have taken part in the research, or they can be simply added on as one publication for each country. The third way is to only add a publication on to the country that the first author in the list represents.

The research of different countries is assessed with both country specific assessments and comparisons between countries. It is harder to notice trends in country specific assessments than with comparisons between countries, and thus comparing countries is more informative.

Country specific assessment

Measuring national productivity with bibliometric indicators has been happening for a long time. Many countries publish national science reports in which the state of the country's research and research infrastructure are analysed using bibliometric indicators. For example, in the United States, the National Science Board publishes the Science and Engineering Indicators report which can be found at http://www.nsf.gov/statistics/seind08/.

The European Commission publishes reports on the level of science and technology in the EU. The publications are available in at http://cordis.europa.eu/indicators/publications.htm.

The Finnish Science and Technology Information Service offers news and up-to-date statistics about Finnish science and innovations in English at http://www.research.fi/en. It contains key statistics and other data on Finnish science and technology. There are also links for more in-depth information: statistical and other publications, documents and databases.

One of the foremost tasks of the Academy of Finland is evaluation: peer review of applications, impact analysis, disciplinary evaluations and reviews of the research system. Since 1997, the Academy of Finland has reviewed the state and quality of scientific research in Finland at three-year intervals. A few disciplinary evaluations are carried out annually. They have a great influence on the valuation and development of the disciplines. Reports of the evaluations are published in the Academy of Finland publication series and are available online at http://www.aka.fi/en-gb/A/Academy-of-Finland/Academy-publications/Publication-series/.

The ScienceWatch website produced by Thomson Reuters includes, in the Country Profiles section, brief glances at the level of research of assorted countries, evaluated using data from the Essential Science Indicators database. One can find the research profiles of these countries from 2008. The countries chosen are the top countries in science and the up and coming ones.

Country comparisons

Different countries are compared according to the amount of publications and the amount of citations publications have received. The scientific activity of countries varies and can be roughly correlated with the economic status of the country. In order to eliminate the differences in the amounts of publications, one can make comparisons using the amount of citations per publication.

The SCImago Journal & Country Rank site produces country specific information on scientific research assessment and indicators on the basis of the Scopus citation data. Information is available from 1996. It allows you to compare a maximum of four countries or disciplines, for example, according to the amount of publications, citation information indicators or according to international affiliations.