Report: That so many studies cannot be reproduced is a “crisis” in science

From the National Association of Scholars (2018) (open access):

This report deals with an epistemic problem, which is most visible in the large numbers of articles in reputable peer-reviewed journals in the sciences that have turned out to be invalid or highly questionable. Findings from experimental work or observational studies turn out, time and again, to be irreproducible. The high rates of irreproducibility are an ongoing scandal that rightly has upset a large portion of the scientific community. Estimates of what percentage of published articles present irreproducible results vary by discipline. Randall and Welser cite various studies, some of them truly alarming. A 2012 study, for example, aimed at reproducing the results of 53 landmark studies in hematology and oncology, but succeeded in replicating only six (11 percent) of those studies. Irreproducibility can stem from several causes, chief among them fraud and incompetence. The two are not always easily distinguished, but The Irreproducibility Crisis deals mainly with the kinds of incompetence that mar the analysis of data and that lead to insupportable conclusions. Fraud, however, is also a factor to be weighed.David Randall and Christopher Welser, “The Irreproducibility Crisis of Modern Science: Causes, Consequences, and the Road to Reform” at National Association of Scholars

They talk a bit about the politicization of the academy as well, though that’s not their main focus:

Many scientists think of themselves as philosopher kings, far superior to those in the “basket of deplorables.” The deplorables have a hard time understanding why scientists are so special, and why they should vote as instructed by them. More than two thousand years ago, Plato, who promoted the ideal of philosopher kings, also promoted the concept of the “noble lie,” a myth designed to persuade a skeptical population that they should be grateful to be ruled by philosopher kings. Our current scientific community has occasionally resorted to the noble lie, a problem that can’t be fixed by better training in statistics. Noble lies are also irreproducible and damage the credibility of science William Happer, “Afterword: The Irreproducibility Crisis of Modern Science: Causes, Consequences, and the Road to Reform” at National Association of Scholars

Um, yes, sires, as it happens, the masses are revolting… Note:  Yes, that was a top American science mag you heard recently, mourning the loss of legacy mainstream media. Last year’s Report on the loss of credibility for science is something you can read about via a link from social media like Blazing Cat Fur.  You can read the report yourself online. You might or might not hear much about it from the narrow funnel of big traditional mainstream media. Fight for your right to go on knowing. See also: Pushback against abandoning “statistical significance” in science

and

Abandon statistical significance, learn to live with uncertainty, scientists demand

Share