Search for rigour
- Successful replication is the best way to confirm the validity of scientific research.
- A string of high-profile problems has raised serious questions about published research.
When Nature surveyed 1,576 researchers in 2016, more than 70% had at some point in their career tried and failed to reproduce another scientist’s experiment, while more than half failed to reproduce their own results. That’s a major problem because replication is a cornerstone of science: a study yields results which are then taken up by others and retested. Successful replication helps confirm validity; failure can suggest further scrutiny is needed. At least, that’s the theory.
“We need to incentivize methodological rigour and robust data analysis over results,” says Michèle Nuijten, an expert in meta-science from Tilburg University in the Netherlands. Unfortunately, headline-grabbing breakthroughs are more attractive to journals, so failed replication studies often go unpublished: only 13% of Nature’s respondents had this type of paper accepted.
Michèle Nuijten, expert in meta-science from Tilburg University
“We need to incentivize methodological rigour and robust data analysis over results.”
Some implicated scientists have felt publicly shamed, others have insinuated incompetence by replicators. But many more see an opportunity to improve data analysis. “Openness is key,” says Nuijten. “Several journals now enforce policies of data sharing, which turns out to be a highly effective way to drive compliance among researchers.” Funders in Nuijten’s homeland have taken the incentive to replicate further. In 2016, the Netherlands Organisation for Scientific Research (NWO) launched the first grants programme dedicated to replication, worth €3 million.
“That’s a relatively small amount,” says Nuijten “but the announcement generated a lot of interest and sent a positive signal that funding agencies may be willing to invest in this vital type of research.”