Opinion

The simple math problem that blows apart the NSA's surveillance justifications

Here's why the NSA dragnet won't work

Here's a question about death and probability, done first by Cory Doctorow. Suppose one out of every million people is a terrorist (if anything, an overestimate), and you've got a machine that can determine whether someone is a terrorist with 99.9 percent accuracy. You've used the machine on your buddy Jeff Smith, and it gives a positive result. What are the odds Jeff is a terrorist?

Try to figure it out, or at least guess, before you read on.

Here's the answer: a 0.1 percent chance — which is to say, the 99.9 percent accurate test will give you the wrong answer 99.9 percent of the time. Seems low, doesn't it?* This is the false positive paradox, and it completely blows up any possible security justification for the NSA's dragnet surveillance of our phone calls and emails.

The issue is stated simply and elegantly by Doctorow: "When you try to find something really rare, your test's accuracy has to match the rarity of the thing you're looking for." If it does not, then the number of false positives will completely bury the signal in irrelevant garbage.

And the numbers I used beforehand were not some crazy extrapolation. If anything, they were far too generous to the likes of the NSA. Their procedure for identifying terror suspects is not anything like a pushbutton machine, and is almost certainly less than 99.9 percent accurate. Instead, it's a colossal hodgepodge that has yet to produce any tangible successes.

Worse, there are no simple follow-up tests that might confirm the result. Instead, agents have to be dispatched to undertake a lengthy investigation, taking weeks or even months. As Bruce Schneier points out in his new book, this exact problem has bedeviled terrorism investigators:

In the years after 9/11, the NSA passed to the FBI thousands of tips per month; every one of them turned out to be a false alarm. The cost was enormous, and ended up frustrating the FBI agents who were obligated to investigate all the tips. We also saw this with the Suspicious Activity Reports —or SAR — database: tens of thousands of reports, and no actual results. And all the telephone metadata the NSA collected led to just one success: the conviction of a taxi driver who sent $8,500 to a Somali group that posed no direct threat to the U.S. — and that was probably trumped up so the NSA would have better talking points in front of Congress. [Data and Goliath]

Indeed, it's arguable that an obsessive focus on dragnet surveillance is actually a distraction from more effective investigative techniques, because even moderately competent terrorists will avoid electronic communication altogether. Bin Laden was suspicious of even encrypted email years before the Snowden leaks, but especially today, one would have to be grossly misinformed to express sympathy for terrorism online. This might explain why the FBI has spent so much time of late baiting utterly hapless chumps or the mentally ill into taking fake weapons and explosives they never would have been able to get on their own.

At any rate, as I've argued before, simple bureaucratic competence and bog-standard detective work are vastly underrated compared to piling up gigantic quantities of irrelevant data. But the false positive problem ought to be the final nail in the dragnet coffin. Unless terrorism becomes thousands of times more common than it is today, such broad techniques will be utterly useless against real terrorism.

*Out of every million people, 1 will be a terrorist, and 1000 (0.1 percent of 1 million) will be false positives. Therefore, Jeff's probability is 1/(1000+1) = 0.001, or 0.1 percent.

Correction: An earlier version of this article incorrectly stated Jeff's probability as 0.01 percent.

More From...

Picture of Ryan CooperRyan Cooper
Read All
Democrats' mini-nuke option to stave off economic collapse
Joe Manchin and Kyrsten Sinema.
Opinion

Democrats' mini-nuke option to stave off economic collapse

The case for medical price fixing
Pills.
Opinion

The case for medical price fixing

Tigray and the shredding of international law
A Tigray child.
Opinion

Tigray and the shredding of international law

The only way to save Biden's agenda
President Biden and Krysten Sinema.
Opinion

The only way to save Biden's agenda

Recommended

Iowa Sen. Chuck Grassley is running for re-election in 2022
Chuck Grassley
Running

Iowa Sen. Chuck Grassley is running for re-election in 2022

Biden's immigration bind
President Biden.
Picture of W. James Antle IIIW. James Antle III

Biden's immigration bind

The Haitian migrant surge, explained
Haitian migrants.
Briefing

The Haitian migrant surge, explained

'Distinctive navy-blue shoes' helped the FBI arrest Jan. 6 Capitol riot suspect
Blue shoes
Capitol Riot Aftermath

'Distinctive navy-blue shoes' helped the FBI arrest Jan. 6 Capitol riot suspect

Most Popular

Jimmy Fallon and Nicole Kidman almost make it through interview without awkwardness
Jimmy Fallon and Nicole Kidman
Last Night on Late Night

Jimmy Fallon and Nicole Kidman almost make it through interview without awkwardness

7 cartoons about America's vaccine fights
Editorial Cartoon.
Feature

7 cartoons about America's vaccine fights

Democrats are governing like Republicans
A donkey.
Picture of W. James Antle IIIW. James Antle III

Democrats are governing like Republicans