Here's a good rule of thumb for interpreting science news


You've probably seen it before: a headline that goes something like "Study: caffeine gives you brain cancer." And you think, What, really? Such results are often followed by a similar headline a few months later with the exact opposite result. How do we sort through this endless stream of science?
Aaron Carroll's most recent video suggests some useful tools for this purpose. He tells us about meta-analyses, which are studies built on previously done studies. By combining data, one can increase statistical power and perhaps uncover new results. Many of the most counterintuitive studies — and therefore most widely reported — are meta-analyses. Here are some tips to keep in mind:
* Conducting a meta-analysis requires a judgment call. One has to pick which studies go in and which do not. This can and should be done on rigorous grounds, but it means that meta-analyses will always be a bit more vulnerable to cherry-picking than a randomized controlled trial.
* Meta-analyses are for weak relationships. If an effect is strong, then it will be easy to find. Smoking's relationship with lung cancer, for example, practically explodes out of the data. Meta-analyses, by contrast, are mostly done on questions where the effect is not nearly so obvious, like whether organically-grown food is more healthy.
Therefore, one should generally approach a new meta-analysis with a wary eye. They're an important tool, but not infallible. Watch the full video explanation below. --Ryan Cooper
Subscribe to The Week
Escape your echo chamber. Get the facts behind the news, plus analysis from multiple perspectives.

Sign up for The Week's Free Newsletters
From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.
From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.
Sign up for Today's Best Articles in your inbox
A free daily email with the biggest news stories of the day – and the best features from TheWeek.com
Ryan Cooper is a national correspondent at TheWeek.com. His work has appeared in the Washington Monthly, The New Republic, and the Washington Post.
-
Film reviews: The Phoenician Scheme, Bring Her Back, and Jane Austen Wrecked My Life
Feature A despised mogul seeks a fresh triumph, orphaned siblings land with a nightmare foster mother, and a Jane fan finds herself in a love triangle
-
Music reviews: Tune-Yards and PinkPantheress
Feature "Better Dreaming" and "Fancy That"
-
Withdrawing 529 plan funds for college? Here's what to know.
the explainer Maximize the amount you have stashed away for your education
-
Breakthrough gene-editing treatment saves baby
speed read KJ Muldoon was healed from a rare genetic condition
-
Sea lion proves animals can keep a beat
speed read A sea lion named Ronan beat a group of college students in a rhythmic dance-off, says new study
-
Humans heal much slower than other mammals
Speed Read Slower healing may have been an evolutionary trade-off when we shed fur for sweat glands
-
Novel 'bone collector' caterpillar wears its prey
Speed Read Hawaiian scientists discover a carnivorous caterpillar that decorates its shell with the body parts of dead insects
-
Scientists find hint of alien life on distant world
Speed Read NASA's James Webb Space Telescope has detected a possible signature of life on planet K2-18b
-
Katy Perry, Gayle King visit space on Bezos rocket
Speed Read Six well-known women went into lower orbit for 11 minutes
-
Scientists map miles of wiring in mouse brain
Speed Read Researchers have created the 'largest and most detailed wiring diagram of a mammalian brain to date,' said Nature
-
Scientists genetically revive extinct 'dire wolves'
Speed Read A 'de-extinction' company has revived the species made popular by HBO's 'Game of Thrones'