It’s a fact of scientific research - not every experiment you conduct will be a success. “Failure” shouldn’t be a dirty word, though. Only by learning from personal failures or those of colleagues can researchers figure out the best ways to solve hard problems. So why is it that so many scholarly publications appear devoid of evidence of research failures?
Scholars have been questioning an apparent trend in positive-outcome bias, as evidenced in the 2011 study published in Scientometrics, “Negative results are disappearing from most disciplines and countries.” The study, which analyzed the results of over 4,600 papers published across disciplines, found that the frequency of scholars reporting positive supports for tested hypotheses has grown 22% between 1990 and 2007, with differences by discipline and country. This report and others like it are raising awareness of the imbalance between positive and negative results in research publications. But, what is causing this trend towards favoring positive outcomes and what should scholars be doing about it?
Authors lack venues to publish null and negative research
According to Eric Martens, Senior Editor at PLOS ONE a primary reason that there is dwindling evidence of null and negative research outcomes in academia is that scholars lack a place to publish them. Preparing manuscripts is of course a time-consuming endeavor, and the effort required to submit even the most significant of null or negative findings may seem futile to scholars if they do not feel confident journals will consider them.
“The dearth of published negative results has always been an issue, in part because of limited incentives for authors, but also because there have been so few journals that publish these studies,” Martens explained. “Journals could help to relieve this pressure by encouraging and facilitating the publication of negative results.”
Since its launch in 2006, Martens said PLOS ONE has always welcomed and encouraged submissions that report negative results, which have traditionally been shunned by publishers and scientists. However, despite the journal’s editorial policies encouraging such submissions, they have received few.
In February 2015 PLOS ONE launched “The Missing Pieces: A Collection of Negative, Null and Inconclusive Results,” to raise awareness of their support of consequential null and negative findings and to make it clear to scholars that there is a place for null and negative research in PLOS.
“Negative” Results has a Negative Connotation
In addition to journals making an effort to seek and publish consequential null and negative research findings, Martens said raising awareness of the value of null and negative results among the scholarly community is key to getting researchers to share them.
“On the author side, we need better incentives to publish negative results. Though carrying out and publishing these studies is beneficial for science as a whole, scholars may not want to dedicate resources to writing, submitting, and revising such studies if they don’t feel the work is valued,” said Martens.
Martens said one of the biggest factors in altering perceptions of the value of null and negative findings is how they are addressed by hiring, tenure, and promotion committees, as well as funders. He said, currently, just as some of these bodies may devalue an article because it is published in a journal with a low impact factor, the same value may not be attributed to a negative result as for one that is positive.
“There are tremendous pressures on academics to first get hired, and then get funding, tenure, and promotion, as well as teach, mentor, and fulfil various other roles. A negative result may be set aside and ignored if it is not clear how publishing that result aligns with surviving in a tremendously competitive field,” Martens explained. “A change in such perceptions could greatly affect researchers’ attitudes.”
Martens said that in his opinion, part of changing how academia perceives the value of null and negative research is focusing on how it is presented. For example, scholars who get a negative result contrary to outcomes predicted by previous literature could present their findings as cause for the research community to reassess old practices or beliefs that current scholarship is being built upon.
“I think taking these results and placing them into a broader context, seeing them as the missing pieces rather than ‘failed’ experiments, will help gain more traction for publishing them overall,” said Martens.
Why should scholars and journals care about null and negative results?
One of the major reasons to publish null and negative results is that only publishing positive outcomes can distort scholarly literature, which can potentially cause biases in scholars expecting or seeing a desired effect.
“Imagine 10 researchers studying a topic where 5 of them observe an effect and 5 do not. If only the studies reporting an effect are published, a subsequent review of the literature would suggest that 100% of researchers agree, whereas the reality is much more complex,” explained Martens.
Another repercussion of not publishing negative results is that resources of time and money are wasted when scholars conduct experiments that have already been disproven unbeknownst to them.
“The literature often points to hypotheses that are appealing and testable by many researchers in a field. If the first of these studies publishes a negative result, it could spare the unnecessary replication of experiments by countless other groups,” said Martens. “This is a prime example of why funders should place greater value on the publication of such studies.”
The stresses scholars face in a positive-biased atmosphere is also reason to consider changing the status quo. If scholars and journals continue to favor positive results, more researchers may put forth positive results that they are unsure of propelled by the pressure to “publish or perish.” There is also the chance of scholars avoiding riskier research endeavors to favor experiments of which they can more accurately predict the outcome.
