Can medical research be brought back from rigor mortis?

A review of Richard F. Harris Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Billions by Marcus Munafò at Nature:

Harris introduces us to the growing field of metascience — the scientific study of science itself — and some of those working in it. These reproducibility firefighters are providing answers to such empirical questions, and identifying interventions. Robert Kaplan and Veronica Irvin at the US National Institutes of Health (NIH) showed that when the National Heart, Lung, and Blood Institute required preregistration of primary outcomes (the main outcome against which success should be judged) in clinical trials, the proportion of studies reporting a benefit fell from 57% to 8%.

That’s the good news. The bad news would have been perpetuating the errors.

Failure is a normal part of science, but dressing it up as success (for example, by presenting a secondary outcome as the primary outcome) is misleading. So is packaging exploratory, hypothesis-generating work as confirmatory, hypothesis-testing work. Unfortunately, with few ways to publish negative results, such practices are encouraged by incentives to present clean results with a compelling narrative, and be the first to do so. More.

Wouldn’t be nice if the Marches and teach-ins were about issues like replication failure?

How many of those people stop and think what it would be like to be fighting cancer and to discover that billions of our tax dollars have been wasted on ineffectual research? Oh and, if we complain, we “hate science.”

Keep up to date with Retraction Watch.

See also: For better science, let’s dismiss the rubbish that “science is self-correcting” in principle Self-correction is a rational human choice; it is not inherent in any enterprise whatsoever.

Peer review “unscientific”: Tough words from editor of Nature

Crisis in replication

and

Marchin’, marchin’ for Science (Hint: the problems are back at your desk, not out in the streets)

 

Share
  • Art Deco

    How many of those people stop and think what it would be like to be
    fighting cancer and to discover that billions of our tax dollars have
    been wasted on ineffectual research? Oh and, if we complain, we “hate science.”

    Research funds may have been suboptimally deployed, but there’s been quite a bit of success in locating effective treatments for cancer since 1970. The ratio of annual deaths to annual diagnoses was 0.63 in 1970; it was 0.36 in 2010. You have eight or nine coarse types of cancer (accounting for perhaps a quarter of all diagnoses) which remain killers more often than not. People generally survive the other sorts (or live to such an age that something else takes them before the cancer has a chance to resurface).

    • But how much better luck would we have had if there had not been so many problems around replication standards, etc.?

  • Hard Little Machine

    It starts with the funding process. Grants are given out to confirm a hypothesis whether it’s a theory or a product or someone else’s research. But the point is to conclude whatever the group putting up the money wants to fund. Research is about grant-grabbing. And let’s be clear – almost ALL research is meta-research. Paying someone to review pre existing data in order to reshape it in some way. Whenever you read about ‘a study of 22,000 patients….etc’ they’re talking about someone else’s compilation of data on something else that happens to contain a few odd artifacts to be looked at anew.

    Peer review is bogus because no one has the time to do it for free unless they have some personal axe to grind. It’s all pretty much bullshit.