Post by @masek@infosec.exchange

2026-10-03 11:06 UTC

In this case, there is also a simple mathematical reason why mass surveillance works like shit. You have three ingredients: a very large population,a tiny number of actual terrorists,and a detection system with a measurable false-positive rate. That combination is mathematically nasty. Assume you monitor one million people and ten of them are actually terrorists. Now assume you have an extraordinarily good detection system: it identifies 90% of the terrorists and has a false-positive rate of only 0.1%. It will correctly identify nine terrorists. But among the 999,990 innocent people, a 0.1% false-positive rate produces roughly 1,000 false alarms. So your supposedly excellent surveillance system gives you about 1,009 alerts, of which only nine are real. More than 99% of your alerts are false positives. And this example is deliberately generous. A false-positive rate of just 0.1% is unrealistically low for a real-world system trying to infer something as vague and complicated as terrorist intent from communications, behavior, associations, travel patterns, social media activity and other surveillance data. At 1%, the same example would produce roughly 10,000 false positives for nine true positives. At 5%, it would produce nearly 50,000. This is the base-rate problem. When the thing you are looking for is extremely rare, even a remarkably accurate detection system can produce vastly more false positives than true positives. In the real world, where detection is messy and the criteria themselves are ambiguous, the problem gets much worse. And now you have created a second problem: alarm fatigue. Someone has to investigate all those innocent people. Analysts learn that almost every alarm they see is bullshit. Attention becomes the scarce resource. The system designed to find the needle in the haystack responds to the difficulty of finding the needle by continuously adding more hay. Mass surveillance therefore has a fundamental scaling problem: collecting more data is easy. Producing more alerts is easy. Producing more human attention is not.

Replies (4)

@masek@infosec.exchange 2026-10-03 11:22

As a side note: during the invasion of Iraq, more than twenty years ago, I used to summarize another problem with counterterrorism like this: Killing one terrorist with a bomb produces ten new terrorists. The number ten was obviously rhetorical, not a statistic. But the underlying mechanism is remarkably similar. Again, you have a selection problem. You are trying to kill a very small number of terrorists embedded in a much larger population of people who are not terrorists. Your weapon and your intelligence are imperfect, so you inevitably produce false positives. Except that in this case, a false positive is not someone who gets investigated. It may be someone who gets killed. And people tend not to react favorably to their children, parents, spouses, friends or neighbors being killed. So the false positives feed back into the system. Every innocent person killed creates a circle of people with a very personal reason to hate whoever dropped the bomb. Most of them will never become terrorists, of course. But you don't need many of them to do so before your counterterrorism operation starts producing some of the very thing it is supposed to eliminate. In surveillance, false positives consume attention. In kinetic counterterrorism, false positives can create enemies. Different consequences, same fundamental problem: when the target is extremely rare and your selection mechanism is imperfect, being wrong vastly more often than being right can dominate the entire system.

Open ##5060547
@IcooIey@mastodon.green 2026-10-03 11:15

@masek@infosec.exchange and, the problems you point out are exploitable by the real terrorists, too, compounding them.

Open ##5060579
@Leszek_Karlik@circumstances.run 2026-10-03 11:24

@masek@infosec.exchange Mass surveillance doesn't work in democratic societies with human rights. It does work, quite efficiently, in fascist societies, because creating random victims through false positives is a feature, not a bug for a fascist dictatorship, instilling fear in the population. (The elites don't have to worry... as long as they're on the right side of the internal power struggles, of course)

Open ##5300179
@angusm@mastodon.social 2026-10-03 12:13

@masek@infosec.exchange Thank you for this clear & useful analysis. I think some decision-makers favor mass surveillance because it fills a psychological need. They want to believe in its effectiveness because then they can persuade themselves they solved the problem. A darker take is that the goals of some advocates of mass surveillance aren’t too different from those of the terrorists: to fragment or reshape society to increase their own power. Mass surveillance is a form of state terrorism.

Open ##5300183