The new and improved Times Higher Education (THE) Impact Rankings 2020 were published this week with as much online fanfare as THE could muster. Unfortunately, they are not improved enough.
My sketchnotes here: https://photos.app.goo.gl/Dj47SHE2ehzdEMM17
“There are limits to what universities can do and the SDGs don’t capture everything about the impact of our research.”
Plus, the measurement is based on journal articles from commercial databases (eg: Scopus). Those databases index have language bias. On the other hand, we are lacking of national level scientific database that provide dataset for those rankings to process.
All rankings measure the following components, which all of them contain level of bias:
These are the rankings that increasingly drive institutional behaviour – and competition between them.
and not to mention it drives external economical-social setting eg: labour market, top university labeling in the mind of parents, etc.
As a result, the THE clings to a methodology that despite taking insufficient account of the false precision and the uncertainties introduced by the proxy nature of the indicators used to ‘measure’ actual performance, still claims to be able to distinguish universities on scores that differ by 0.1%. It is laughable to claim this level of precision. It is to universities’ discredit that they go along.
For less economically stable countries (eg Indonesia), many indicators are very much controlled by national level situations (regulations, funding), geographical settings, and the large sum of high school graduates to enter undergraduate degree. On the contrary, all rankings only relevant for graduate research.