Evaluation based on scientific publishing: Ranking lists

About ranking lists

The evaluation criteria of university ranking lists are different. Methodology, used indicators and data on which indicators are based on vary as well as the weighting of indicators in evaluations. In the subject rankings or field rankings attention should be paid to the fact that the classification of publications to the disciplines is not congruent in the databases like Scopus and Web of Science. Besides bibliometrics other evaluation criteria are the ratio of students and staff, the quality of teaching and reputation surveys. Results between ranking lists should not be compared. Benchmarking should only be done between different years in regard to the same ranking.

Some of the ranking lists evaluate only universities, some also  research institutions and hospitals.

Methodologies of the some most famous university ranking lists:

Rankings that are based just on bibliometrics:

  • CWTS Leiden Ranking
  • National Taiwan University Ranking
  • University Ranking by Academic Performance, URAP

Rankings that use bibliometrics and also other methods:

  • Center for World University Rankings, CWUR
  • European Research Ranking
  • QS World University Rankings
  • Ranking Web of World Universities, Webometrics
  • SCImago Institutions Rankings
  • Shanghai Ranking eli Academic Ranking of World Universities, ARWU
  • Times Higher Education World University Rankings,THE
  • U.S. News & World Report's Best Global Universities Rankings
  • U-Multirank

​Rankings offering also subject/field rankings:

  • Center for World University Rankings, CWUR
  • CWTS Leiden Ranking
  • National Taiwan University Ranking, NTU
  • QS World University Rankings
  • Shanghai Ranking eli Academic Ranking of World Universities (ARWU)
  • Times Higher Education World University Rankings
  • U.S. News & World Report's Best Global Universities Rankings  
  • University Ranking by Academic Performance, URAP

Reading:

Data sources of publication information of ranking lists

Scopus
  • QS World University Rankings 
  • SCImago Institutions Rankings project (SIR) 
  • Times Higher Education World University Ranking (THE) (from 2015)  

SCimago/Scopus

  • Webometrics, Ranking Web of World Universities
Web of Science
  • Center for World University Rankings, CWUR
  • CWTS Leiden Ranking 
  • National Taiwan University Ranking, NTU
  • Shanghai Ranking  Academic Ranking of World Universities (ARWU)  
  • U-Multirank
  • U.S. News & World Report's Best Global Universities Rankings 
  • URAP-ranking

JCR/Web of Science

  • Center for World University Rankings, CWUR

Other databases and resources

  • Ahrefs: Webometrics, Ranking Web of World Universities
  • CORDIS: European Research Ranking
  • Google Scholar: Webometrics, Ranking Web of World Universities
  • Majestic: Webometrics, Ranking Web of World Universities
  • PATSTAT: SCImago Institutions Rankings project (SIR) 
  • PlumX: SCImago Institutions Rankings project (SIR) 
  • Unpaywall: SCImago Institutions Rankings project (SIR) ​

Shanghai Ranking of World Universities, ARWU

The most well know ranking list of world universities is the Shanghai Academic Ranking of World Universities (ARWU) published by ShanghaiRanking Consultancy. Rankings for universities and subjects available.

Methodology 

"ARWU uses six objective indicators to rank world universities, including the number of alumni and staff winning Nobel Prizes and Fields Medals, number of highly cited researchers selected by Clarivate Analytics, number of articles published in journals of Nature and Science, number of articles indexed in Science Citation Index - Expanded and Social Sciences Citation Index, and per capita performance of a university. More than 1800 universities are actually ranked by ARWU every year and the best 1000 are published."

Times Higher Education World University Rankings, THE

Times Higher Education World University Rankings is published by  Times Higher Education journal. From 2015 the publication data is based on Scopus and also books and book articles are taking into account in evaluation. University and subject rankings available.

2020 methodology

Weighting of the indicators:

"The performance indicators are grouped into five areas: Teaching 30% (the learning environment); Research 30% (volume, income and reputation); Citations 30%  (research influence); International outlook 7,5% (staff, students and research); and Industry Income 2,5% (knowledge transfer)."

QS World University Rankings

QS World University Rankings is published by Quacquarelli Symonds. Publication data is based on Scopus. University and subject rankings available.

Weighting of evaluation criteria:

  1. Academic Reputation 40%
  2. Employer Reputation 10%
  3. Faculty/Student Ratio 20%
  4. Citations per faculty 20 %
  5. International Faculty Ratio 5%
  6. International Student Ratio 5%

CWTS Leiden Ranking

CWTS Leiden Ranking uses bibliometrics based on Web of Science. Only articles and reviews articles are taken into account. University and subject rankings available.

Indicators:

  • Publications
  • Size-dependent vs. size-independent indicators
  • Scientific impact indicators
  • Collaboration indicators
  • Open access indicators
  • Gender indicators

National Taiwan University Ranking, NTU

National Taiwan University Ranking  uses bibliometrics in evaluation. University and subject rankings available. Publication data is based on  Clarivate's Essential Science Indicators (ESI).  Methodology and  indicators

Weighting of indicators:

  • research productivity 25 %
  • research impact 35%
  • research excellence 40%

University Ranking by Academic Performance, URAP

URAP-rankings is published by  Informatics Institute of Middle East Technical University. Ranking list is based on Web of Science data and the methodology is bibliometrics. University and field rankings available.

Indicators:

  • articles 21%
  • citations 21%
  • total documents 10%
  • article impact total 18%
  • international collaboration 15%

SCImago Institutions Rankings

SCImago Institutions Rankings project (SIR) is developed by SCImago Research Group. Ranking is based on:

"...research performance, innovation outputs and societal impact."

Data sources:

  • Scopus
  • PATSTAT (patentit)
  • Google and Ahrefs (näkyvyys)
  • Unpaywall (julkaisujen avoimuus)
  • PlumX  (altmetriikka)
  • Mendeley (altmetriikka)

Weighting of the criteria:

  • Research 50%
  • Innovation 30%
  • Societal 20%

SIR-methodology

U-Multirank

U-Multirank is an European multidimensional system for benchmarking universities and research institutes:

"...is not confined to research but takes into account different aspects and dimensions of the performance of universities: teaching and learning, research, knowledge transfer, international orientation and regional engagement."

"...does not produce a combined, weighted score across these different areas of performance and then use these scores to produce a numbered league table of the world’s “top” 100 universities. The underlying principle is that there is no theoretical or empirical justification for such composite scores...Therefore, the U-Multirank methodology looks at the scores of universities on individual indicators and places these in five performance groups  which are very good, good, average, below average, weak."

U-Multirank partners:

Centre for Higher Education (CHE)
Center for Higher Education Policy Studies (CHEPS)
Centre for Science and Technology Studies (CWTS)
Fundación CYD

Methodology and data sources

Center for World University Rankings, CWUR

CWUR, Center for World University Rankings evaluation method:

"...assesses the quality of education, alumni employment, research output, and citations without relying on surveys and university data submissions."

Evaluation criteria:

  • Quality of education 25%
  • Alumni employment 25%
  • Quality of Faculty 10%
  • Research Perfomance
    • Research Output 10%
    • High-Quality Publications 10%
    • Influence 10%
    • Citations 10%

Data source for publications is  Journal Citation Reports, JCR which is based on Web of Science.

U.S. News & World Report's Best Global Universities Rankings

 

U.S. News & World Report's Best Global Universities Rankings collaborates with Clarivate's InCites, based on   Web of Science. University and subject rankings available.

Indicators:

  • Global research reputation 12.5%
  • Regional research reputation 12.5%
  • Publications 10%
  • Books 2.5%
  • Conferences 2.5%
  • Normalized citation impact 10%
  • Total citations 7.5%
  • Number of publications that are among the 10% most cited 12.5%
  • Percentage of total publications that are among the 10% most cited 10%
  • International collaboration – relative to country 5%
  • International collaboration 5%
  • Number of highly cited papers that are among the top 1% most cited in their respective field 5%
  • Percentage of total publications that are among the top 1% most highly cited papers 5%

European Research Ranking

European Research Ranking uses data of the CORDIS-database.

Methodology is based on:

  • Funding and project participation performance
  • Networking activity and alliances
  • Diversity of research areas

Ranking Web of World Universities, Webometrics

Webometrics or Ranking Web of World Universities - published by the Cybermetrics Lab - uses both web-metrics (webometric) and bibliometrics in evaluations

 "...It uses both webometric (all missions) and bibliometric (research mission)  indicators..."

Published twice a year and rankings also for research institutes, hospitals and repositories.

Methodology and indicators:

  • Presence 5%
  • Visibility 50%
  • Transparency (or openness) Top cited researchers 10%
  • Excellence (or scholar) Top cited papers 35%

Data sources:

  • Google
  • Ahrefs
  • Majestic
  • Scimago

Good practices in institutional web positioning