Research Assessment at EMBL

EMBL supports fair and responsible research assessment practices, which include its recruitment and performance assessment processes.

EMBL’s commitment to responsible research assessment (RRA)

EMBL is committed to outstanding research, service, and administration performance by hiring and training the best personnel in a culture that values and promotes equality, diversity, and inclusion.

We recognise the value of a broad range of scientific outputs, including research articles, data, and software.
We assess research outputs and other research contributions based on their intrinsic merit.
We discourage the inappropriate use of proxies or metrics – such as the impact factor of the journal in which the work was published.

Responsible research assessment at EMBL

EMBL aims to play a leading role in driving changes to the research assessment processes to make it fairer and more transparent. It became a signatory of DORA in 2018 and convened a working group in 2020. In 2022, EMBL became a signatory of the Coalition on Advancing Research Assessment (CoARA).

EMBL’s Responsible Research Assessment (RRA) Working Group plays a key role in designing and implementing initiatives to support the practices recommended by DORA and CoARA. Thus far, the Working Group has developed guidelines for EMBL’s practices for recruitment, performance review, and promotion. The Working Group is now implementing and monitoring the progress of our research assessment efforts.

What are DORA and CoARA?

The San Francisco Declaration on Research Assessment (DORA) was developed in 2012 with the goal of improving the ways in which the output of scientific research is evaluated by funding agencies, academic institutions, and other parties. Its objectives range from raising awareness of new tools and processes in research assessment to catalysing change in research assessment reform. DORA signatories can be either individuals or organisations. EMBL is proud to be one of DORA’s Case Studies.

The Coalition for Advancing Research Assessment (CoARA) was formally launched in December 2022 with more than 300 signatory organisations to its agreement. The agreement is a result of collaboration with the European Commission, Science Europe, the European University Association, and others that began in January 2022. The CoARA agreement aims to maximise the quality and impact of research by setting a shared direction and shared commitments and timelines for changes in assessment practices.

Implementing responsible research assessment at EMBL

EMBL has developed a set of Research Assessment Recommendations and is implementing processes to uphold these principles consistently across its research areas.

Our Research Assessment Recommendations are:

To be more explicit to candidates on EMBL selection criteria, all job advertisements include a statement on EMBL’s commitment to DORA.
To ensure all research outputs are considered during assessment, candidates for recruitment, promotions, and awards are asked to include the full range of research outputs in their CVs and to describe the significance of key research outputs in a narrative within their CVs (see below).
To ensure that EMBL committees engaged in recruitment, research performance assessments, and promotion evaluations are adequately briefed on EMBL’s research assessment practices, committee members and reviewers are provided with a guide that guides them to assess scientific outputs rather than journal-based metrics of the publication platform.
EMBL is committed to a culture of content-based research assessment. The way EMBL researchers talk about their, or their colleagues’, research achievements needs to embody the DORA and CoARA spirit to emphasise content quality over publication venues.

EMBL CV instructions

To ensure all research outputs are considered during assessment, candidates for recruitment, promotions, and awards are asked to include a variety of research outputs in their CVs and to describe the significance of key research outputs in a narrative within the CV.

As well as listing publications, research outputs can include open data sets, databases, code, software, pre-prints, patents, commercial products, instruments, clinical practice developments, educational products, policy publications, and any other relevant outputs to which you have contributed.

At the beginning of the list of research outputs, we specifically request a brief narrative (max. 300 words) summarising the impact and importance of the main research outputs (citing outputs by a persistent identifier if available). Please explain your role in contributing to the generation of new ideas and hypotheses and which key skills you have used. You can also highlight how you have communicated your ideas and research results (written and verbally), as well as any related prizes or awards you have received. Please avoid mention of impact factors or H indexes. You may consider a broad range of impact measures directly related to the research output item, including qualitative indicators of research impact such as influence on policy and practice.

Frequently asked questions

What are the current research assessment practises that RRA is trying to improve?

There is a growing consensus amongst scientists, science funders, and policy makers that commonly used standards for assessing research and researchers may not capture the full breadth of scientific research outputs.

Oftentimes, academics and organisations rely on and prioritise journal impact factors, H-indices, and the number of publications or grants received to evaluate researchers and the quality of research. These methods of assessment create skewed incentives and put pressure on the research community to succeed in a small number of performance metrics, rather than the full range of valuable scientific contributions. Moreover, impact factors have been shown to be unreliable measures of citations and do not apply to research outputs other than publications.

How is RRA being implemented at EMBL?

EMBL Research Assessment Recommendations were developed by an institute-wide working group, co-chaired by Jo McEntyre and Wolfgang Huber. The Working Group includes members from across EMBL, including Research, Services, Training, Human Resources, and Strategy.

Does this mean EMBL discourages publications in Nature, Science, Cell, etc.?

No. EMBL Research Assessment Recommendations are only concerned with how we at EMBL discuss the value and impact of research. They discourage talking about impact factors as a proxy for high-quality science, and instead focus on the specific merits of the work in question.

How were the CV instructions developed?

The EMBL CV Instructions were developed during a dedicated workshop hosted by the working group titled ‘Fairer research assessment practices at EMBL’. The workshop included representatives from different career paths and stages, as well as from different EMBL sites. You can find the workshop report here.

How do I know what to include in my research output narrative and in my cover letter?

The research output narrative should summarise the impact and importance of your main research outputs. A cover letter should explain why your skills and previous experience are a good fit for the specific position/programme you are applying to. We encourage you to peruse this guide to academic cover letters (pdf) produced by the EMBL Fellows’ Career Service.

How will my research output narrative be assessed?

Together with its CV instructions, EMBL has developed recommendations for those reading and assessing CVs for recruitment, internal evaluation, and promotion. We ask all recruitment and assessment panels to assess the research outputs described on their intrinsic merit, and to consider the relevance of the work described to the advancement of the field. We discourage the inappropriate use of proxies or metrics during assessment processes – such as the title or impact factor of the journal in which the work was published.

How many research outputs should I discuss in my research output narrative?

As long as your overall narrative is within 300 words, this is up to you. The number of research outputs you choose to discuss will depend on your specific field of study and on your career stage. Senior researchers in some fields may wish to include more outputs, while more junior researchers may have fewer items they wish to include.

Explore

Open Science at EMBL

For a positive culture change in life science research

Equality, Diversity and Inclusion

Creating a more inclusive research and work culture

EMBL Missions

EMBL, a centre of excellence for Europe's leading molecular biologists, pursues ﬁve missions