bdschobel

From today's Los Angeles Times (summarized):

Do the Math: Rooting Out Terrorists Is Tricky Business

By John Allen Paulos (professor of mathematics at Temple University)

...Upon detecting supposedly telltale patterns, law enforcement would hope to stop pre-perpetrators before they commit crimes. It's a worthy goal, but in pursuing it the government will collect, integrate and evaluate extensive personal data on all of us, greatly compromising our privacy and perhaps even our political liberty. Is it worth the cost to society?

Let's consider a mathematical approach to that question, one that derives from probability theory and the obvious fact that the vast majority of people of every ethnicity are not terrorists.

For the sake of argument, let's assume that eventually some system of information gathering and interpretation becomes so uncannily accurate that when it examines a future terrorist (someone with terrorist intentions), 99% of the time it will correctly identify him as a pre-perpetrator. Furthermore, when this system examines somebody who is harmless, 99% of the time the system will correctly identify him as harmless. In short, it makes a mistake only once every 100 times.

Now let's say that law enforcement apprehends a person using this technology. Given these assumptions, one might guess that the person would almost certainly be a terrorist. Right? Well, no. Even with the system's amazing data-mining powers, there would be only a tiny chance that the apprehended person would have gone on to commit a terrorist act if he had not been caught.

To see why this is so and to make the calculations easy, let's postulate a population of 300 million people of whom 1,000 are future terrorists. The system will correctly identify, we're assuming, 99% of these 1,000 people as future terrorists. Thus, since 99% of 1,000 is 990, the system will apprehend 990 future terrorists. Great.

But wait. There are, by assumption, 299,999,000 nonterrorists in our population, and the system will be right about 99% of them as well. Another way of saying this is that it will be wrong about 1% of these people. Since 1% of 299,999,000 equals 2,999,990, the system will swoop down on these 2,999,990 innocent people as well as on the 990 guilty ones, apprehending them all.

That is, the system will arrest almost 3 million innocent people, about 3,000 times the number of guilty ones. And that occurs, remember, only because we're assuming the system has these amazing powers of discernment. If its powers are anything like our present miserable predictive capacities, an even greater percentage of those arrested will be innocent.



[This message has been edited by bdschobel (edited 01-19-2003).]

bdschobel

Here's a link to the story (protecting copyright):

http://www.latimes.com/news/opinion/...,4188924.story

You have to register to read it.

Bruce

[This message has been edited by bdschobel (edited 01-19-2003).]

richard

This is quite a good analysis of a very serious paradox.

That is, if a test gives 99% reliable results showing a "positive" (for whatever is being tested, say a disease or for terrorism), but only a tiny percentage of the test subjects have the disease or are terrorists or whatever, then if you test positive chances are overwhelming that you do not have the disease or are not a terrorist.

Years ago I tested positive for lupus and was very depressed for awhile. A repeat test was run and showed that I didn't have the disease. I was a victim of this "paradox."

I think those of us who are concerned about privacy should rally around a new monicker.

I suggest

Security with Privacy.

Sort of a "peace with honor" thing. It has a ring to it. The point is that we need to be as secure as possible while respecting the privacy of each of us and defending our private lives against government intrusion.

Spiff

A nicely put mathematical explanation of the harassment that goes on at airports today and why "better safe than sorry" is an expression that is a steaming load of crap.

------------------
"Give me Liberty or give me Death." - Patrick Henry

runningshoes

This article, or at least the portion posted above, is a perfect example of why everyone should go through at least one class of advanced statistics and one class in critical/logical thinking.

First of all, in any similar statistical analysis where you're trying to find a group with a certain characteristic, your main population that you're pulling from (let's use security screening as an example) is one and the same. What this means is that, for arguments sake, we're talking about 50 million flyers per year (with a huge % of this group flying more than once per year).

Now, while all the 50MM get profiled in the system (go through security), only a portion get flagged by the system for security purposes (secondary screening). Let's assume 1 million (again my number) get pulled for secondary due to profiles that have even the slightest connection as a potential terrorist, possible Visa issues, country of origin, cash tickets, yada, yada, yada.

Out of this million, the system would find X who are future terrorists, in his case 990 people, while missing 10. On the flip side, his assumption of a "Type B" mistake yields almost 10,000 false readings. I have no idea where his leap to an arrest for each false reading came out, and when you use actual numbers for your screened population, the rations are completely different than what the article states.

It's completely unprofessional to publish this type of BS in a paper under the guise of a scientific analysis.

FWAAA

As I've been saying since September 13, 2001, or so, our government is failing us by acting as though all 280 million of us are terrorists (or potential terrorists) and then clearing us only when we are sharp and pointy object free.

Problem is, our system doesn't stop (or even attempt to identify) the terrorists. Instead, we boosted the pay of the sharp and pointy object searchers and then declared victory.

Somewhere, in that haystack of passengers, could be lurking a terrorist or two (probably not many more), yet our TSA (and our new TSA FT friends) continue to do nothing about that possibility. One even posted that he/she didn't want any alarms and wanted nothing more than to move us annoyances thru the checkpoint so they would have more time to chat.

That's my problem with the whole premise behind the TSA. Its whole solution is to slow down the checkpoints (as is almost always the case where I travel), annoy non-terrorist Americans, and ultimately do nothing to identify and stop real terrorists from flying. What a crappy solution. I expect more from the US government.

NoStressHere

Just additional information that shows how foolish we are being. Why not take one tenth of one percent of what we are spending and put it into research and development. Put some really smart people that are not elected into a room for a few weeks and study this whole thing. Okay, they should spend a week at some major airports as well to "discover" some obvious weaknesses.

I really believe that intelligent review would show how foolish the entire process is. Get the mgmt away from the press and the politicians and see what comes of it.

EPS

Using intelligence about intelligence
http://sfgate.com/cgi-bin/article.cg...0/DD165361.DTL

tsadude

The people who work the trenches have made several observations and recommndations but they have been ignored.

bdschobel

That really bothers me, and I mean that very sincerely. Please e-mail me your suggestions, and watch what I do with them. They will not be ignored. I don't go to Washington once a week for nothing.

Bruce

tsadude

There is nothing I would like better but I cannot afford to take a chance like that. Revealing these things could be really ugly and send my ... to jail for discussing it here or in an email message.

bdschobel

Your identity would NEVER be revealed. Actually, I don't know who you are, anyway!

Bruce

Xyzzy

You could always go down to your local library and sign up for a hotmail account and/or use an anonymizer and then send him your thoughts. That, coupled with the fact that nobody here knows who you are or even if you really work for the TSA, should provide enough anonymity.

There *are* ways to get the information transferred quietly and secretly if you want to take the time to do it.

mdtony

Someone needs to get out of his ivory tower and into the real world.

tsadude

What we need is a secure line of communication beyond our FSDs. There are many of us who have tried alot of different things to smooth out the screening process and have been successful. We do not really care who takes credit, but at least listen to what we have to say.