Setting one benchmark for universities is hard.

Setting 27 benchmarks is a Herculean task – but after three years of poring over measurement and data as the statistical expert on the COKI team, it is a task which Research Fellow Dr Chun-Kai (Karl) Huang is taking in his stride.

The COKI project has developed the most comprehensive database of open knowledge indicators in the world, drawing insights from more than 12 trillion data elements to shed new light on the role and impact of universities in the 21^st century.

With such a huge reservoir of information to hand, it’s clear that existing rankings, based on limited data sets are distorted – largely ignoring whole continents and presenting a picture based on a limited range of paywall-protected journal data. Indeed, the team have found that it is possible to significantly increase the status of an individual and a discipline within a university simply by locating missing citations and papers that were previously omitted from the data used for various rankings.

With a goal of understanding the role and contribution of universities in the 21^st century, and who they allow to make knowledge and to learn it, the COKI team have cast a much broader net, examining data with a far broader inclusion of institutions in large ignored areas of South America and Africa, as well as measures which will provide new insights into inclusion, access and impact.

Dr Huang is currently working on a fundamental paper underpinning the theory behind this work, looking at 27 indicators that the COKI team have collected data on, and examining their efficacy and value in pinpointing aspects of university performance in relation to open knowledge.

“We collected data about a year ago that has evolved around open knowledge institutions,” Dr Huang said.

“There are a whole bunch of indicators, diversity, open access, website score, annual report analysis, analysis on policies, crossref events and collaboration measures – each of which can potentially contribute significant insights into university performance.

“While the individual measures are fascinating, it’s really the combination of these indicators which will yield the richest insights about the open knowledge potential and performance of an institution.

“This data could be used to help identify university strengths, impact, and relevance, as well as areas where the university faces some challenges.

“Curtin supported the COKI project because the University was committed to making positive change and wanted to understand how universities could better serve communities in the 21^st century.

“This next paper aims to be an important foundation for discussions around what the best form of a university can look like – how it can be measured and what impact it will have on its community. We can’t possibly master all 27 indices in this paper, but it will provide a clear outline of what measures are most reliable, and also where data may appear to be misleading, and needs considerable context to be properly understood.

“Given the changes in the university sector at the moment, this is a really fascinating field to be working in at the moment, and we are looking forward to further discussions with colleagues internationally being sparked once the paper is complete.”