The simple math problem that blows apart the NSA's surveillance justifications

Here's a question about death and probability, done first by Cory Doctorow. Suppose one out of every million people is a terrorist (if anything, an overestimate), and you've got a machine that can determine whether someone is a terrorist with 99.9 percent accuracy. You've used the machine on your buddy Jeff Smith, and it gives a positive result. What are the odds Jeff is a terrorist?

Try to figure it out, or at least guess, before you read on.

Here's the answer: a 0.1 percent chance — which is to say, the 99.9 percent accurate test will give you the wrong answer 99.9 percent of the time. Seems low, doesn't it?* This is the false positive paradox, and it completely blows up any possible security justification for the NSA's dragnet surveillance of our phone calls and emails.

Subscribe to The Week

Escape your echo chamber. Get the facts behind the news, plus analysis from multiple perspectives.

SUBSCRIBE & SAVE

Sign up for The Week's Free Newsletters

From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.

From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.

Sign up

The issue is stated simply and elegantly by Doctorow: "When you try to find something really rare, your test's accuracy has to match the rarity of the thing you're looking for." If it does not, then the number of false positives will completely bury the signal in irrelevant garbage.

And the numbers I used beforehand were not some crazy extrapolation. If anything, they were far too generous to the likes of the NSA. Their procedure for identifying terror suspects is not anything like a pushbutton machine, and is almost certainly less than 99.9 percent accurate. Instead, it's a colossal hodgepodge that has yet to produce any tangible successes.

Worse, there are no simple follow-up tests that might confirm the result. Instead, agents have to be dispatched to undertake a lengthy investigation, taking weeks or even months. As Bruce Schneier points out in his new book, this exact problem has bedeviled terrorism investigators:

Indeed, it's arguable that an obsessive focus on dragnet surveillance is actually a distraction from more effective investigative techniques, because even moderately competent terrorists will avoid electronic communication altogether. Bin Laden was suspicious of even encrypted email years before the Snowden leaks, but especially today, one would have to be grossly misinformed to express sympathy for terrorism online. This might explain why the FBI has spent so much time of late baiting utterly hapless chumps or the mentally ill into taking fake weapons and explosives they never would have been able to get on their own.

At any rate, as I've argued before, simple bureaucratic competence and bog-standard detective work are vastly underrated compared to piling up gigantic quantities of irrelevant data. But the false positive problem ought to be the final nail in the dragnet coffin. Unless terrorism becomes thousands of times more common than it is today, such broad techniques will be utterly useless against real terrorism.

*Out of every million people, 1 will be a terrorist, and 1000 (0.1 percent of 1 million) will be false positives. Therefore, Jeff's probability is 1/(1000+1) = 0.001, or 0.1 percent.

Correction: An earlier version of this article incorrectly stated Jeff's probability as 0.01 percent.