Research Assessment

Body

Definition

Researchers and research projects deliver a wide range of valuable scientific results along the scientific discourse. This can include data sets, a new element of analysis software, teaching material, video documentations, new technologies and procedures and many other research artefacts. Yet, in today's academic assessment practices, evaluations are primarily, or sometimes exclusively, focused on scientific papers. This sets misguiding incentives, can lead to inflationary publication practices and discourages open science practices. The problem is known for many years and was clearly described in the 2013 DORA declaration. However, only recently, concrete attempts are made to reform academic research practices.

The outputs from scientific research are many and varied, including: research articles reporting new knowledge, data, reagents, and software; intellectual property; and highly trained young scientists. Funding agencies, institutions that employ scientists, and scientists themselves, all have a desire, and need, to assess the quality and impact of scientific outputs. It is thus imperative that scientific output is measured accurately and evaluated wisely.

DORA - San Francisco Declaration on Research Assessment (2013)

 

Goals at CERN

CERN is committed to collaborating with its Member and Associate Member States to incentivise quality-assured and reusable Open Science practices for research and career assessments. As part of this effort, CERN strives to support strategic discussions and implementation of community-specific rewards systems, such as responsible metrics covering the full diversity of the community’s research outputs, moving away from primarily publication-based assessment practices. 

 

Services and activities

CERN is actively supporting the reform of research assessment both at the policy level as well as providing tools and services to enable new practices. This includes but is not limited to the following examples:

CoARA

Through a global Coalition of research funding organisations, research performing organisations, national/regional assessment authorities and agencies, as well as learned societies and other relevant organisations, signatories of the Agreement on Reforming Research Assessment will work together to enable systemic reform on the basis of common principles within an agreed timeframe, and to facilitate exchanges of information and mutual learning between all those willing to improve research assessment practices. Based on 10 commitments, the coalition will establish a common direction for research assessment reform, while respecting organisations’ autonomy. The Agreement sets a shared direction for changes in assessment practices for research, researchers and research-performing organisations, with the overarching goal to maximise the quality and impact of research. It includes the principles, commitments and timeframe for reforms and lays out the principles for a Coalition of organisations willing to work together in implementing the changes.

CERN has signed the agreement as one of 51 early signatories and became a member of CoARA at its constituting assembly in December 2022. Based on the timelines and milestones defines in the Agreement, and with the direct involvement of researchers across all career stages, CERN will review existing practices and implement a revised research assessment framework.

INSPIRE

INSPIRE is a trusted community hub that helps researchers to share and find accurate scholarly information in high-energy physics (HEP). It serves as a one-stop information platform for HEP community, comprising eight interlinked databases on literature, researchers, experiments, conferences, institutions, journals, jobs, and seminars. Run in collaboration by CERN, DESY, Fermilab, IHEP, IN2P3, and SLAC, it has been serving the scientific community for almost 50 years. Previously known as SPIRES, it was the first website outside Europe and the first database on the web. Close interaction with the user community and with arXiv, ADS, HEPData, ORCID, PDG and publishers is the backbone of INSPIRE’s evolution. The INSPIRE collaboration has signed the DORA declaration in full support of transparent and holistic research assessment.

The provided services help already today to assess researchers independent of journal publications by creating and counting transparent citations between pre-prints and published articles. To create further incentives of FAIR software and dataset releases and respective citations, INSPIRE will include two more databases in its service suite: research datasets and research software. Supporting the DORA and CoARA principles, this will allow a holistic assessment of a diverse set of scientific contributions and supports moving away from a journal-centric output analysis.

Zenodo

Zenodo was built and developed by researchers already 10 years ago, to ensure that everyone can join in Open Science. The OpenAIRE project, in the vanguard of the open access and open data movements in Europe was commissioned by the EC to support their nascent Open Data policy by providing a catch-all repository for EC-funded research. 

To fully understand and reproduce research performed by others, it is necessary to have all the details. In the digital age, this means a variety of digital artefacts such as preprints, software, datasets, videos, analysis notes, etc. Zenodo hosts almost 3 millions records across all these different research products. To be an effective catch­-all, that eliminates barriers to adopting data-sharing practices, Zenodo does not impose any requirements on format, size, access restrictions or licence. 

Zenodo assigns DOIs to all records and helps researchers receive credit by making the research results citable and through OpenAIRE integrates them into existing reporting lines to funding agencies like the European Commission. Citation information is also passed to DataCite and onto the scholarly aggregators supporting an inclusive research assessment across the globe.

 

Key contacts at CERN