̽��ֱ�� of Cambridge - open research

Cambridge ̽��ֱ�� signs San Francisco Declaration on Research Assessment

cjb250 — Mon, 08 Jul 2019 08:09:15 +0000

DORA’s recommendations call for institutions not to use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles when assessing researchers’ contributions in hiring, promotion, or funding decisions. It encourages universities, researchers and others to assess research on its own merits rather than on the basis of the journal in which the research was published and highlights the need to capitalise on the opportunities provided by online publication.

Professor Chris Abell, Pro-Vice Chancellor for Research at Cambridge, said: “ ̽��ֱ�� ̽��ֱ�� of Cambridge is committed to producing excellent research. By signing up to DORA, we want to demonstrate to our researchers that we value the quality and content of their research regardless of how and where it is published.”

Professor Steve Russell from the ̽��ֱ��’s Department of Genetics, will chair the DORA Working Group, which will oversee the implementation of the DORA recommendations.

“This is an important step for the ̽��ֱ��, particularly for early career researchers where all too often career progression is based on judgments using flawed metrics,” says Professor Russell. “By signing DORA the ̽��ֱ�� is making very positive step towards developing a culture where research excellence is assessed by the quality of the work and not by the title of the Journal where it is published.”

DORA calls on institutions to be explicit about the criteria used to reach hiring, tenure, and promotion decisions, clearly highlighting, especially for early-stage researchers, that the content of a paper is much more important than publication metrics or the identity of the journal in which it was published.

In addition to research publications, DORA recommends considering the value and impact of all research outputs (including datasets and software) and a broad range of impact measures including qualitative indicators of research impact, such as influence on policy and practice, for the purposes of research assessment.

̽��ֱ�� ̽��ֱ��’s HR Division will begin implementing a number of changes to ensure the agreement’s recommendations are reflected across its recruitment, reward and promotions schemes.

Brigitte Shull, Director of Scholarly Communications Research & Development at Cambridge ̽��ֱ�� Press, added: “ ̽��ֱ��principles of DORA align with our open research strategy and ongoing activities around improved metrics and recognizing author contributions. By signing up to DORA, we want to help improve the way the quality of research is assessed and expand the range of tools to better account for a variety of research outputs.”

In February, Cambridge became one of the first UK universities to publish a position statement on Open Research. Its statement set out the key principles for the conduct and support of Open Research at the ̽��ֱ��, which aims to increase inclusivity and collaboration, unlock access to knowledge and improve the transparency and reproducibility of research.

̽��ֱ��recommendation to sign DORA at the ̽��ֱ�� was made by the Open Research Working Group, chaired by Professor Richard Penty, and at the Press by the Open Research Steering Committee, chaired by Brigitte Shull.

̽��ֱ�� ̽��ֱ�� of Cambridge and Cambridge ̽��ֱ�� Press announced on 8 July 2019 that they have signed up to the San Francisco Declaration on Research Assessment (DORA), a set of recommendations agreed in 2012 that seek to ensure that the quality and impact of research outputs are 'measured accurately and evaluated wisely'.

By signing up to DORA, we want to demonstrate to our researchers that we value the quality and content of their research regardless of how and where it is published

Chris Abell

kkolosov

Microscope

̽��ֱ��text in this work is licensed under a Creative Commons Attribution 4.0 International License. Images, including our videos, are Copyright © ̽��ֱ�� of Cambridge and licensors/contributors as identified. All rights reserved. We make our image and video content available in a number of ways – as here, on our main website under its Terms and conditions, and on a range of channels including social media that permit your use and sharing of our content under their respective Terms.

Yes

Licence type:

Public Domain

6,000 and counting: Cambridge Vice-Chancellor joins Stephen Hawking in making his PhD ‘Open Access’

sjr81 — Wed, 27 Feb 2019 17:00:09 +0000

Cambridge also becomes first UK university to publish position statement on Open Research.

Opinion: ̽��ֱ��science ‘reproducibility crisis’ – and what can be done about it

cjb250 — Mon, 20 Mar 2017 09:57:15 +0000

A survey by Nature revealed that 52% of researchers believed there was a “significant reproducibility crisis” and 38% said there was a “slight crisis”.

We asked three experts how they think the situation could be improved.

Open Research is the answer

Danny Kingsley, head of the Office of Scholarly Communication, ̽��ֱ�� of Cambridge

̽��ֱ��solution to the scientific reproducibility crisis is to move towards Open Research – the idea that scientific knowledge of all kinds should be openly shared as early as it is practical in the discovery process. We need to reward the publication of research outputs along the entire process, rather than just each journal article as it is published.

As well as other research outputs – such as data sets – we should reward research productivity itself as well as the thought process and planning behind the study. This is why Registered Reports was launched in 2013, where researchers register the proposal and how the research will be conducted, before any experimental work commences. It allows editorial decisions to be based on the rigour of the experimental design and increases the likelihood that the findings could be replicated.

In the UK there is now a requirement from most funders that the data underpinning a research publication is made available. However, although there are moves towards open research, many argue against the sharing of data among the research community.

Questionable findings are often hidden. Shutterstock

Researchers often write multiple papers from a single data set and many fear that if this data is released with the first publication then the researcher will be “scooped” by another research group, who will publish findings from similar data sets before the original authors get the chance to publish follow up articles – to gain maximum credit for the work. If the publication of data itself could be recorded as a “research output”, then being scooped would no longer be such an issue, as such credit will have been given.

One benefit of sharing data could be an improvement in its quality – as previous research has shown. And there have been small steps towards this goal, such as a standard method of citing data.

We also need to publish “null” results – those that do not support the hypothesis – to prevent other researchers wasting time repeating work. There are a few publication outlets for this, and a recent press release from ResearchGate indicated that it supports the sharing of failed experiments through its “project” offering. It lets users upload and track experiments as they are happening – meaning no one knows how they will turn out.

Psychology is leading the way out of crisis

Jim Grange, senior lecturer in psychology, Keele ̽��ֱ��

To me, it is clear that there is a reproducibility crisis in psychological science, and across all sciences. Murmurings of low reproducibility began in 2011 – the “year of horrors” for psychology – with a high profile fraud case. But since then, ̽��ֱ��Open Science Collaboration has published the findings of a large-scale effort to closely replicate 100 studies in psychology. Only 36% of them could be replicated.

̽��ֱ��incentive structures in universities and the attitude that you “publish or perish” means that researchers prioritise “getting it published” over “getting it right”. It also means that some, implicitly or explicitly, use questionable research practices to achieve publication. These may include failing to report parts of data sets or trying different analytical approaches to make the data fit what you want to say. It could also mean presenting exploratory research as though it was originally confirmatory (designed to test a specific hypothesis).

However, many psychology journals now recommend or require the preregistration of studies which allow researchers to detail their predictions, experimental protocols, and planned analytical strategy before data collection. This provides confidence to readers that no questionable research practices have occurred.

Erasing data: a questionable research practice. Shutterstock

Registered Reports has taken this further. But of course, once results are produced, isolated findings don’t mean much until they have been replicated.

I make efforts to replicate results before trying to publish and you’d be forgiven for thinking that replication attempts are common in science, but this is simply not the case. Journals seek novel theories and findings, and view replications as treading over old ground which offers little incentive for career-minded academics to conduct replications.

This has also led to the introduction of Registered Replication Reports in Perspectives on Psychological Science. This is where teams of researchers each follow identical procedures independently and aim to replicate important findings from the literature. A single paper then collates and analyses them to establish the size and reproducibility of the original study.

Although psychology is leading the way for improvements with these pioneering initiatives, it is certainly not out of the woods. But it has started to move beyond a crisis and make impressive strides – more disciplines need to follow suit.

This is a publication bias crisis

Ottoline Leyser, director of the Sainsbury Laboratory, ̽��ֱ�� of Cambridge

Reproducibility is a fundamental building block of science. If two people do the same experiment, they should get the same result. But there are many good reasons why two “identical” experiments might not give the same result such as unknown differences that have not been considered – and some exciting discoveries have been made this way.

So if a lack of reproducibility is itself not necessarily a problem, why is everybody talking about a crisis? In some cases poor practice and corner cutting have contributed to lack of reproducibility, and there have been some high profile cases of out and out fraud. It’s a major concern, but what is causing it?

In 2014 I chaired a project on the research culture in Britain for the Nuffield Council on bioethics, which was motivated by concerns about research integrity including over-claiming, rushing prematurely to publication and incorrect use of statistics. ̽��ֱ��main conclusions were that poor practice is incentivised by hyper-competition with overly narrow rules for winning.

There is an excessive focus on the publication of groundbreaking results in prestigious journals. But science cannot only be groundbreaking, as there is a lot of important digging to do after new discoveries – but there is not enough credit in the system for this work and it may remain unpublished because researchers prioritise their time on the eye-catching papers, hurriedly put together.

̽��ֱ��reproducibility crisis is actually a publication bias crisis which is driven by the reward structures in the research system. Various approaches have been suggested to address problems, such as pre-registration of experiments. However, the research landscape is highly diverse and this type of solution is only sensible for some research types. ̽��ֱ��most widely relevant solution is to change the reward structures. In the UK there is a major opportunity to do this by reforming the Research Excellence Framework (REF). Through the REF, public money is allocated to universities based on the “quality” of the four best research outputs, usually papers, produced by each of their principal investigators over approximately six years and it disproportionately rewards groundbreaking research.

We need reward for a portfolio of research outputs, including not only the headline grabbing results, but also confirmatory work and community data sharing, which are the hallmarks of a truly high quality research endeavour. This would go a long way to shifting the current destructive culture.

Ottoline Leyser, Director of the Sainsbury Laboratory, ̽��ֱ�� of Cambridge; Danny Kingsley, Head, Office of Scholarly Communication, ̽��ֱ�� of Cambridge, ̽��ֱ�� of Cambridge, and Jim Grange, Senior Lecturer in psychology, Keele ̽��ֱ��

This article was originally published on ̽��ֱ��Conversation. Read the original article.

Reproducibility is the idea that an experiment can be repeated by another scientist and they will get the same result. It is important to show that the claims of any experiment are true and for them to be useful for any further research. However, science appears to have an issue with reproducibility.

NIAID

Study of Human Immune Response to HIV

̽��ֱ��text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes

Licence type:

Attribution