Here's a good rule of thumb for interpreting science news
You've probably seen it before: a headline that goes something like "Study: caffeine gives you brain cancer." And you think, What, really? Such results are often followed by a similar headline a few months later with the exact opposite result. How do we sort through this endless stream of science?
Aaron Carroll's most recent video suggests some useful tools for this purpose. He tells us about meta-analyses, which are studies built on previously done studies. By combining data, one can increase statistical power and perhaps uncover new results. Many of the most counterintuitive studies — and therefore most widely reported — are meta-analyses. Here are some tips to keep in mind:
* Conducting a meta-analysis requires a judgment call. One has to pick which studies go in and which do not. This can and should be done on rigorous grounds, but it means that meta-analyses will always be a bit more vulnerable to cherry-picking than a randomized controlled trial.
* Meta-analyses are for weak relationships. If an effect is strong, then it will be easy to find. Smoking's relationship with lung cancer, for example, practically explodes out of the data. Meta-analyses, by contrast, are mostly done on questions where the effect is not nearly so obvious, like whether organically-grown food is more healthy.
Therefore, one should generally approach a new meta-analysis with a wary eye. They're an important tool, but not infallible. Watch the full video explanation below. --Ryan Cooper
The Week
Escape your echo chamber. Get the facts behind the news, plus analysis from multiple perspectives.
Sign up for The Week's Free Newsletters
Join more than 350,000 subscribers and keep yourself informed with a selection of The Week’s most interesting, enlightening and entertaining stories - plus daily puzzles.
Join more than 350,000 subscribers and keep yourself informed with a selection of The Week’s most interesting, enlightening and entertaining stories - plus daily puzzles.
A free daily email with the day’s biggest news and analysis – and the best features from TheWeek.com
Ryan Cooper is a national correspondent at TheWeek.com. His work has appeared in the Washington Monthly, The New Republic, and the Washington Post.
Explore your latest briefing
-
-
talking pointsLivestreaming an execution: cruelty or added deterrent? -
QUOTE OF THE DAY‘You don’t get to take a man from his family. You don’t get to take his very life, his whole future from him, and just walk away from it. Especially when that man is Tupac Shakur.’
-
today’s big questionCan a Democrat win the Texas governor’s race?
-
Explore your latest briefing
-
-
talking pointsLivestreaming an execution: cruelty or added deterrent? -
QUOTE OF THE DAY‘You don’t get to take a man from his family. You don’t get to take his very life, his whole future from him, and just walk away from it. Especially when that man is Tupac Shakur.’
-
today’s big questionCan a Democrat win the Texas governor’s race?
-