Health research reaches the public through a chain — study, press release, news article, social media — and at each stage certainty tends to increase while caveats fall away.
A few questions applied to any health headline will filter most of the noise.
Was it in humans
The first and most productive question. A large proportion of dramatic health headlines describe studies in cells or in animals.
These are legitimate science and an essential stage in understanding mechanisms. The translation rate to human clinical effect is low — most compounds showing promise in preclinical work do not go on to demonstrate benefit in people.
A headline about a substance killing cancer cells is describing something that a great many substances do in a dish, including bleach. What matters is whether it does so in a person, at a tolerable dose, better than existing treatment.
Association or intervention
The second question and the source of most misleading coverage.
Observational studies find that people who do one thing have different outcomes from people who don't. They cannot establish that the first caused the second, because the groups differ in countless other ways.
The classic failure: people who take a supplement are also more likely to exercise, have higher incomes, and see doctors more often. Any of those could produce the observed difference.
Researchers adjust for known confounders and can only adjust for what they measured. Unmeasured confounding is a permanent limitation.
Randomised trials solve this by allocating the intervention randomly, which balances both known and unknown differences. When a randomised trial contradicts an observational finding, the trial is generally the more reliable — and this has happened repeatedly, including for several widely promoted supplements.
The vocabulary tells you which you're reading. "Linked to", "associated with" and "correlated with" indicate observational work. "Reduced", "caused" and "prevented" imply intervention, and are frequently applied incorrectly to observational findings by the time a headline is written.
Relative or absolute
The most common way a small effect gets made to sound large.
A doubling of risk sounds alarming. If the baseline risk was one in ten thousand, the new risk is two in ten thousand — an absolute increase of one in ten thousand.
Both figures are accurate. One is useful for deciding whether to change your behaviour and the other isn't.
Relative figures are more dramatic, which is why they dominate coverage. If a headline gives a percentage change without a baseline, you cannot tell whether the finding matters.
How many people
Small studies produce unstable results. A trial with thirty participants can produce a striking finding that disappears in a larger one.
Small studies also have low statistical power, which counterintuitively means that any significant finding they do produce is more likely to be an overestimate.
This is a substantial contributor to the pattern of dramatic findings that fail to replicate.
What was actually measured
Surrogate outcomes versus outcomes people care about.
A drug that improves a blood marker has demonstrated an effect on that marker. Whether it reduces heart attacks is a separate question, and there are historical examples of treatments that improved a surrogate while increasing mortality.
Similarly, a study finding improved scores on a questionnaire has found improved questionnaire scores. Whether that corresponds to a difference people would notice depends on the scale and the magnitude.
Who paid for it
Industry funding doesn't invalidate research, and it's associated with results favouring the sponsor across multiple fields.
The mechanisms are subtle rather than fraudulent — comparator choice, dose selection, outcome definition, and decisions about publication. All defensible individually and collectively they shift results.
Disclosure statements are usually at the end of a paper and worth reading.
Is it new, or is it noise
Single studies rarely change what's known. Fields advance through accumulation, and any individual result sits in a distribution.
Systematic reviews and meta-analyses, which pool multiple studies, are considerably more informative than any single trial and receive a fraction of the coverage because they aren't news.
Which suggests a general rule: if a finding contradicts an established body of evidence, the prior probability that it's wrong is high. That's not closed-mindedness, it's appropriate weighting.
The practical filter
For most health headlines: was it in humans, was it randomised, how many people, what was the absolute effect, and does it agree with what was already known.
Five questions, and they'll tell you within a minute whether a story is worth further attention. Most aren't, which is the honest and slightly deflating conclusion.
General information only. Health decisions should be discussed with a qualified healthcare professional.
Where to look instead
Some practical sources that are more reliable than general news coverage.
Systematic review databases, which pool evidence rather than reporting single studies, and which state their methods explicitly.
National health service information pages, which are written for the public, updated against guidelines, and have no commercial interest.
Clinical guideline documents themselves. Denser than news coverage and they state the strength of evidence behind each recommendation, which is exactly the information that gets stripped out elsewhere.
And the original paper's abstract, which is usually freely available and frequently more measured than the coverage of it. Reading the limitations paragraph takes two minutes and is where researchers say what they actually think.