You've probably seen it before: a headline that goes something like "Study: caffeine gives you brain cancer." And you think, What, really? Such results are often followed by a similar headline a few months later with the exact opposite result. How do we sort through this endless stream of science?
Aaron Carroll's most recent video suggests some useful tools for this purpose. He tells us about meta-analyses, which are studies built on previously done studies. By combining data, one can increase statistical power and perhaps uncover new results. Many of the most counterintuitive studies — and therefore most widely reported — are meta-analyses. Here are some tips to keep in mind:
* Conducting a meta-analysis requires a judgment call. One has to pick which studies go in and which do not. This can and should be done on rigorous grounds, but it means that meta-analyses will always be a bit more vulnerable to cherry-picking than a randomized controlled trial.
* Meta-analyses are for weak relationships. If an effect is strong, then it will be easy to find. Smoking's relationship with lung cancer, for example, practically explodes out of the data. Meta-analyses, by contrast, are mostly done on questions where the effect is not nearly so obvious, like whether organically-grown food is more healthy.
Therefore, one should generally approach a new meta-analysis with a wary eye. They're an important tool, but not infallible. Watch the full video explanation below. --Ryan Cooper
The Week
Escape your echo chamber. Get the facts behind the news, plus analysis from multiple perspectives.
Sign up for The Week's Free Newsletters
From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.
From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.
-
Violent videos of Charlie Kirk’s death are renewing debate over online censorshipTalking Points Social media ‘promises unfiltered access, but without guarantees of truth and without protection from harm’
-
What led to Poland invoking NATO’s Article 4 and where could it lead?TODAY'S BIG QUESTION After a Russian drone blitz, Warsaw’s rare move to invoke the important NATO statute has potentially moved Europe closer to continent-wide warfare
-
Africa could become the next frontier for space programsThe Explainer China and the US are both working on space applications for Africa
-
SpaceX breaks Starship losing streak in 10th testspeed read The Starship rocket's test flight was largely successful, deploying eight dummy satellites during its hour in space
-
Rabbits with 'horns' sighted across Coloradospeed read These creatures are infected with the 'mostly harmless' Shope papilloma virus
-
Lithium shows promise in Alzheimer's studySpeed Read Potential new treatments could use small amounts of the common metal
-
Scientists discover cause of massive sea star die-offSpeed Read A bacteria related to cholera has been found responsible for the deaths of more than 5 billion sea stars
-
'Thriving' ecosystem found 30,000 feet underseaSpeed Read Researchers discovered communities of creatures living in frigid, pitch-black waters under high pressure
-
New York plans first nuclear plant in 36 yearsSpeed Read The plant, to be constructed somewhere in upstate New York, will produce enough energy to power a million homes
-
Dehorning rhinos sharply cuts poaching, study findsSpeed Read The painless procedure may be an effective way to reduce the widespread poaching of rhinoceroses
-
Breakthrough gene-editing treatment saves babyspeed read KJ Muldoon was healed from a rare genetic condition
