Mathematician Criticizes "Security" Program
 

From today's Los Angeles Times (summarized):

Do the Math: Rooting Out Terrorists Is Tricky Business

By John Allen Paulos (professor of mathematics at Temple University)

...Upon detecting supposedly telltale patterns, law enforcement would hope to stop pre-perpetrators before they commit crimes. It's a worthy goal, but in pursuing it the government will collect, integrate and evaluate extensive personal data on all of us, greatly compromising our privacy and perhaps even our political liberty. Is it worth the cost to society?

Let's consider a mathematical approach to that question, one that derives from probability theory and the obvious fact that the vast majority of people of every ethnicity are not terrorists.

For the sake of argument, let's assume that eventually some system of information gathering and interpretation becomes so uncannily accurate that when it examines a future terrorist (someone with terrorist intentions), 99% of the time it will correctly identify him as a pre-perpetrator. Furthermore, when this system examines somebody who is harmless, 99% of the time the system will correctly identify him as harmless. In short, it makes a mistake only once every 100 times.

Now let's say that law enforcement apprehends a person using this technology. Given these assumptions, one might guess that the person would almost certainly be a terrorist. Right? Well, no. Even with the system's amazing data-mining powers, there would be only a tiny chance that the apprehended person would have gone on to commit a terrorist act if he had not been caught.

To see why this is so and to make the calculations easy, let's postulate a population of 300 million people of whom 1,000 are future terrorists. The system will correctly identify, we're assuming, 99% of these 1,000 people as future terrorists. Thus, since 99% of 1,000 is 990, the system will apprehend 990 future terrorists. Great.

But wait. There are, by assumption, 299,999,000 nonterrorists in our population, and the system will be right about 99% of them as well. Another way of saying this is that it will be wrong about 1% of these people. Since 1% of 299,999,000 equals 2,999,990, the system will swoop down on these 2,999,990 innocent people as well as on the 990 guilty ones, apprehending them all.

That is, the system will arrest almost 3 million innocent people, about 3,000 times the number of guilty ones. And that occurs, remember, only because we're assuming the system has these amazing powers of discernment. If its powers are anything like our present miserable predictive capacities, an even greater percentage of those arrested will be innocent.



[This message has been edited by bdschobel (edited 01-19-2003).]

Here's a link to the story (protecting copyright):

http://www.latimes.com/news/opinion/...,4188924.story

You have to register to read it.

Bruce

[This message has been edited by bdschobel (edited 01-19-2003).]

This is quite a good analysis of a very serious paradox.

That is, if a test gives 99% reliable results showing a "positive" (for whatever is being tested, say a disease or for terrorism), but only a tiny percentage of the test subjects have the disease or are terrorists or whatever, then if you test positive chances are overwhelming that you do not have the disease or are not a terrorist.

Years ago I tested positive for lupus and was very depressed for awhile. A repeat test was run and showed that I didn't have the disease. I was a victim of this "paradox."

I think those of us who are concerned about privacy should rally around a new monicker.

I suggest

Security with Privacy.

Sort of a "peace with honor" thing. It has a ring to it. The point is that we need to be as secure as possible while respecting the privacy of each of us and defending our private lives against government intrusion.

A nicely put mathematical explanation of the harassment that goes on at airports today and why "better safe than sorry" is an expression that is a steaming load of crap.

------------------
"Give me Liberty or give me Death." - Patrick Henry

This article, or at least the portion posted above, is a perfect example of why everyone should go through at least one class of advanced statistics and one class in critical/logical thinking.

First of all, in any similar statistical analysis where you're trying to find a group with a certain characteristic, your main population that you're pulling from (let's use security screening as an example) is one and the same. What this means is that, for arguments sake, we're talking about 50 million flyers per year (with a huge % of this group flying more than once per year).

Now, while all the 50MM get profiled in the system (go through security), only a portion get flagged by the system for security purposes (secondary screening). Let's assume 1 million (again my number) get pulled for secondary due to profiles that have even the slightest connection as a potential terrorist, possible Visa issues, country of origin, cash tickets, yada, yada, yada.

Out of this million, the system would find X who are future terrorists, in his case 990 people, while missing 10. On the flip side, his assumption of a "Type B" mistake yields almost 10,000 false readings. I have no idea where his leap to an arrest for each false reading came out, and when you use actual numbers for your screened population, the rations are completely different than what the article states.

It's completely unprofessional to publish this type of BS in a paper under the guise of a scientific analysis.

As I've been saying since September 13, 2001, or so, our government is failing us by acting as though all 280 million of us are terrorists (or potential terrorists) and then clearing us only when we are sharp and pointy object free.

Problem is, our system doesn't stop (or even attempt to identify) the terrorists. Instead, we boosted the pay of the sharp and pointy object searchers and then declared victory.

Somewhere, in that haystack of passengers, could be lurking a terrorist or two (probably not many more), yet our TSA (and our new TSA FT friends) continue to do nothing about that possibility. One even posted that he/she didn't want any alarms and wanted nothing more than to move us annoyances thru the checkpoint so they would have more time to chat.

That's my problem with the whole premise behind the TSA. Its whole solution is to slow down the checkpoints (as is almost always the case where I travel), annoy non-terrorist Americans, and ultimately do nothing to identify and stop real terrorists from flying. What a crappy solution. I expect more from the US government.

Just additional information that shows how foolish we are being. Why not take one tenth of one percent of what we are spending and put it into research and development. Put some really smart people that are not elected into a room for a few weeks and study this whole thing. Okay, they should spend a week at some major airports as well to "discover" some obvious weaknesses.

I really believe that intelligent review would show how foolish the entire process is. Get the mgmt away from the press and the politicians and see what comes of it.

Using intelligence about intelligence
http://sfgate.com/cgi-bin/article.cg...0/DD165361.DTL

The people who work the trenches have made several observations and recommndations but they have been ignored.

That really bothers me, and I mean that very sincerely. Please e-mail me your suggestions, and watch what I do with them. They will not be ignored. I don't go to Washington once a week for nothing.

Bruce

There is nothing I would like better but I cannot afford to take a chance like that. Revealing these things could be really ugly and send my ... to jail for discussing it here or in an email message.

Your identity would NEVER be revealed. Actually, I don't know who you are, anyway! http://www.flyertalk.com/forum/smile.gif

Bruce

You could always go down to your local library and sign up for a hotmail account and/or use an anonymizer and then send him your thoughts. That, coupled with the fact that nobody here knows who you are or even if you really work for the TSA, should provide enough anonymity.

There *are* ways to get the information transferred quietly and secretly if you want to take the time to do it.

Someone needs to get out of his ivory tower and into the real world.

What we need is a secure line of communication beyond our FSDs. There are many of us who have tried alot of different things to smooth out the screening process and have been successful. We do not really care who takes credit, but at least listen to what we have to say.

Let us extend the logic to 99.9% accuracy, and apply it to 50 mil PAX (aasumption that each airport entry is a separate event, i. e. X is not a terrorist today, but could become one tomorrow).

999 of 1000 terrorists, hijackers, etc. would be identified along with with 50,000 innocent PAX

2% of the people detained (140 detained per day)would be correctly selected and 1 terrorist incident could still occur.

I think we all know the answer to that. The screening circus,far more visible than Research and Development, makes a number of travellers and the majority of non-travelling people "feel" safe and gives them an illusion of security.

This thread has been moved to a new board recently setup in TravelBuzz to address security related topics:

Please use the link below to enter the new board:

http://www.flyertalk.com/travel/fttr...e=5&LastLogin=

Radiocycle
In The News Moderator

This thread makes me laugh. If we applied statistics and math to everything, nothing would ever happen..
would you get married? heck no, 50% fail
would you drive? heck no, too many wrecks
would you play the slots? heck no 98% lose.
would you smoke?
would you drink?
would you do anything?

Mathematicians love to make arguments with numbers, fact is, real life doesn't work that way. Gee wait, there have been no hijackings since the TSA took over..that proves things are safer, doesn't it! Eureka!
http://www.flyertalk.com/travel/fttr...m/rolleyes.gif

Holy Cow! I'm a mathematician so I thought I'd steer clear of this thread, but this post got my goat (whatever that means).

married - yes 50% is good odds at this

drive - yes, the % of wrecks is NOT "too many"

slots - sure I play, I count on losing though if you play long enough

smoke - I have, but I know the risk

drink - I have, but I know the risk

anything - yes, including die eventually.

Properly done, you can TRY TO apply math and statistics to everything. In fact one of the most fascinating and beautiful academic endeavors is to apply mathematics to understanding mathematics itself. (See Kurt Godel on this.)

The first post is self evident. A small incidence percentage appllied to a large universe generates many false positives. How can anyone dispute this?

Real mathematics works, false applications of it don't.

How many hijacks were there in the U.S. before 9/11?

It amazes me how little some people feel mathematics plays in their life when I look around and feel surrounded by it http://www.flyertalk.com/travel/fttr...orum/smile.gif

Tazi said:
Depends on how far back you want to go. Lots of them up to 1992, most to Cuba. If anyone wants to read an extract that details a general history of aircraft hijackings since 1931, here is a pretty good one:

http://pegasus.cc.ucf.edu/~surette/hijacking.html

Then after 1992, I'm aware of one (a cargo plane out of Memphis) and then of course the big four.

Of course you are probably well aware my comment was a tongue-in-cheek one and was a bit like the comments made about how certain actions keep Elephants away and that the absence of Elephants (where none ever appear anyway) is evidence of the efficacy of the action. My point was that numbers often are used to prove points when they have no real relationship to reality. At the same time, I am smart enough to recognize that numbers can be important in our life. Have a stress free day.
http://www.flyertalk.com/travel/fttr...m/rolleyes.gif

Thanks, that was my point as well, perhaps not as eloquently said. The question is as you responded to the various scenarios, do we want to take the risks or not? It is probable that any system designed to capture criminals will result in "false positives." Some folks use that percentage as a basis for saying that the process should be eliminated. The question for me is, what are the consequences of no system and do the adverse consequences of that outweigh the "bad" within a system that has false positives. Again, some people have zero tolerance for false positives, others have zero tolerance for a system that allows more criminal acts. Problem is, those who espouse trashing the system have few if any alternatives other than, "I'll take my chances." Fine, but some of us don't want to take that chance. I'd rather have a slightly flawed system where the legal process can work out the details of the flaws rather than no system at all that invites more hijackings. Especially since the terrorists have made it clear that their intent is to do more of the same.

http://www.flyertalk.com/travel/fttravel_forum/cool.gif

Actually, that is most of them up until 1972 not 1992. So, for almost 30 years, there were very few hijackings in the U.S. until 9/11. Well, I hardly think you can use the fact that there have been none since 9/11 to imply it is because of the search for pointy objects being carried out by the TSA since then.

I like how you selectively trimmed what I said to make it look like I said something else.
http://www.flyertalk.com/travel/fttr...um/biggrin.gif

I didn't trim anything. You misquoted the article. Even after reading the whole thing and still not seeing where there had been hijackings to Cuba up until 1992, I did a search of the article. 1992 is not in it. http://www.flyertalk.com/travel/fttr...m/rolleyes.gif



[This message has been edited by tazi (edited 01-26-2003).]

Thanks for this truly thoughtful reply. I guess that's it: some are hopeful the security will help and don't care about the intrusions, others aren't so hopeful and feel the intrusions a waste.

Maybe it is all about optimism vs. cynicism. I think I lean toward the cynical.

REALITY CHECK

The last time I looked, mathematics was an wxact science. Try this for exact. Question 1. Since the implementation of the TSA methods of screening, how many acts of terrorism have been attempted on our airlines? Answer 1. Pick any number. Question 2. How many acts of terrorism have been accomplished on our airlines in that time? Answer 2. 0 Now divide Answer 1 by answer 2 and you get 0%. And 0% is a very nice number.

MATHEMATICAL CHECK

Division by zero is undefined. You get no answer when you divide a number by 0. http://www.flyertalk.com/travel/fttr...orum/smile.gif

[This message has been edited by tazi (edited 01-29-2003).]

1) There is no proof (proofs are very important mathematically. Have you ever done one?) that the TSA is the reason there haven't been any terrorist acts since 9/11. I have a rock that I claim keeps terrorists away. There is just as much specious evidence that my rock keeps terrorists away as there is evidence that the TSA does.

2) As tazi pointed out, your attempts of terrorism vs. successful terrorism ratio is just another example of how mathematically unsound your agency is. You'll need to examine L'Hospital's Rule if you want to even attempt to divide something approaching 0 by something else approaching 0. However, what the TSA really needs is a good course in Probability Theory, for its actions are a complete disgrace when one considers what the probability of a hijacking taking place with some items the idiocy of the TSA attempts to ban.

PS: Welcome to FlyerTalk!

------------------
"Give me Liberty or give me Death." - Patrick Henry


[This message has been edited by Spiff (edited 01-29-2003).]

REALITY CHECK,

Institute the SAME rules regarding box cutters and knives PRIOR to 9/11 and more than likely there is no hijacking. But, they would have found another way to attack the US.

The TSA has done NOTHING to improve Air Travel Security. It is 100,000 Government Employees worth of Window Dressing.

Yiiiiiiiiiiikkkkkkkkkkkes!

The two responses to your post so far were really kind and only pointed out the lack of causality in your analysis.

I'll be crueler and point out that you've got your fraction upside down. I think you want to divide answer 2 by answer 1, not the other way around.

There has been nothing reported to the public. That does not mean any attempts have not been tried.

I'm sure this is no surprise to you, but if your agency continues to cowardly hide behind this arrogant 'need to know' basis for releasing information, your agency will continue to lose credibility with each attempt to hide behind this miserable excuse. If the TSA disgracefully continues in this manner, I sincerely hope Congress will wake up and disenfranchise it.

------------------
"Give me Liberty or give me Death." - Patrick Henry

[This message has been edited by Spiff (edited 02-02-2003).]

Do you really believe that a government agency that deals with security should just release all their info to the public? Think about it for a minute. What would happen if they just gave up all the intel they collected, all their operating procedures, all the why's and how's of the organization. Let's just arm every terrorist in the world, why don't we.

Sorry, but I have failed to see where any intelligence has been used in the screening process so far. I hardly think that the number of pointy objects found should be considered and intelligence leak. Especially when you consider that one airport is selling confiscated items on eBay.

I never said that the number of pointy objects found was an intelligence leak. There is so much more to it than that. The TSA is not just composed of screeners. There are Intelligence Specialists, Criminal Investigators, Law Enforcement Officers, and the list goes on. Eventually the TSA will conduct airline cargo screening, railway security...etc. There are processes and procedures that need to be protected, here.



[This message has been edited by tmspa (edited 02-02-2003).]

Buckle your seatbelts...and you better! Sounds like at some point TSA will be in charge of seatbelt enforcement along with grocery-cart inspection (based on "secure" info that terrorists don't buckle up and load up on junk food just before any terroristic activity.)

Calgon...take me away from the TSA.

When a terminal is evacuated, there better be a @#$% good reason for doing so, and there must be an explanation to all affected - not "it's for your own good." I am not a small child and I do not accept such whitewashing and neither will the rest of the traveling public.



------------------
"Give me Liberty or give me Death." - Patrick Henry

Let's start with your last sentence. Bin Laden and Saddam Hussein were supported by the US govt. in the past so I'm sure some of their arms HAVE come from the US. Not anymore though, so let's move on.

I think if the TSA foiled a terrorist act, no matter how indirectly, it would be a press release in 5 minutes. The TSA is not the CIA. I don't see it as much more than an inefficient feel-good scanning group.

I don't think the TSA does much research, analysis, or actual creative planning work. I'd be shocked if the TSA had an employee at arrivals drop-off scanning cars for suspicious activity.

I'd also be shocked if they, for example, had cameras picking up the license plate of every vehicle that enters the airport. And I know the perimeters of these airports are like swiss cheese. Those maniac snipers near Washington DC probably could have brought down a plane a day with some larger fire power fixed up to their vehicle.

Is TSA scouting the fences around the airport. Is there camera coverage of the airport perimeter. I doubt it. But please continue to take away my kids' mini-swiss army knife and search my 86-year-old Dad.

Oh, and how come TSA allows skate boards on airplanes. Split in thirds the long way (easy if pre-scored) they're clubs. What's up with this stupid exception. And they're wielded by athletic young men exclusively.