Here's a good rule of thumb for interpreting science news


You've probably seen it before: a headline that goes something like "Study: caffeine gives you brain cancer." And you think, What, really? Such results are often followed by a similar headline a few months later with the exact opposite result. How do we sort through this endless stream of science?
Aaron Carroll's most recent video suggests some useful tools for this purpose. He tells us about meta-analyses, which are studies built on previously done studies. By combining data, one can increase statistical power and perhaps uncover new results. Many of the most counterintuitive studies — and therefore most widely reported — are meta-analyses. Here are some tips to keep in mind:
* Conducting a meta-analysis requires a judgment call. One has to pick which studies go in and which do not. This can and should be done on rigorous grounds, but it means that meta-analyses will always be a bit more vulnerable to cherry-picking than a randomized controlled trial.
* Meta-analyses are for weak relationships. If an effect is strong, then it will be easy to find. Smoking's relationship with lung cancer, for example, practically explodes out of the data. Meta-analyses, by contrast, are mostly done on questions where the effect is not nearly so obvious, like whether organically-grown food is more healthy.
Therefore, one should generally approach a new meta-analysis with a wary eye. They're an important tool, but not infallible. Watch the full video explanation below. --Ryan Cooper
Subscribe to The Week
Escape your echo chamber. Get the facts behind the news, plus analysis from multiple perspectives.

Sign up for The Week's Free Newsletters
From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.
From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.
Sign up for Today's Best Articles in your inbox
A free daily email with the biggest news stories of the day – and the best features from TheWeek.com
Ryan Cooper is a national correspondent at TheWeek.com. His work has appeared in the Washington Monthly, The New Republic, and the Washington Post.
-
Local elections 2025: where are they and who is on course to win?
The Explainer Reform UK predicted to make large gains, with 23 councils and six mayoralties up for grabs
By The Week UK Published
-
What is Starmer's £33m plan to smash 'vile' Channel migration gangs?
Today's Big Question PM lays out plan to tackle migration gangs like international terrorism, with cooperation across countries and enhanced police powers
By Sorcha Bradley, The Week UK Published
-
Quirky hot cross buns to try this Easter
The Week Recommends Creative, flavourful twists on the classic Easter bake, from tiramisu and stem ginger to a cheesy sharing-size treat
By Irenie Forshaw, The Week UK Published
-
Dark energy may not doom the universe, data suggests
Speed Read The dark energy pushing the universe apart appears to be weakening
By Peter Weber, The Week US Published
-
Pharaoh's tomb discovered for first time in 100 years
Speed Read This is the first burial chamber of a pharaoh unearthed since Tutankhamun in 1922
By Peter Weber, The Week US Published
-
Scientists report optimal method to boil an egg
Speed Read It takes two temperatures of water to achieve and no fancy gadgets
By Peter Weber, The Week US Published
-
Europe records big leap in renewable energy
Speed Read Solar power overtook coal for the first time
By Peter Weber, The Week US Published
-
Blue Origin conducts 1st test flight of massive rocket
Speed Read The Jeff Bezos-founded space company conducted a mostly successful test flight of its 320-foot-tall New Glenn rocket
By Peter Weber, The Week US Published
-
US won its war on 'murder hornets,' officials say
Speed Read The announcement comes five years after the hornets were first spotted in the US
By Peter Weber, The Week US Published
-
Dark energy data suggest Einstein was right
Speed Read Albert Einstein's 1915 theory of general relativity has been proven correct, according to data collected by the Dark Energy Spectroscopic Instrument
By Peter Weber, The Week US Published
-
New DNA tests of Pompeii dead upend popular stories
Speed Read An analysis of skeletal remains reveals that some Mount Vesuvius victims have been wrongly identified
By Peter Weber, The Week US Published