Heh heh heh.

True. I would be more accurate in saying my part of DOJ hardly ever loses. :D

Sorry twinkie, I thought it would be understood the numbers came from the linked article from the poster I quoted. (We are still doing the baked goods nicknames right?)

http://www.usatoday.com/travel/fligh...etection_N.htm

I also used a percentage calculator located at http://www.csgnetwork.com/csgpercent.html

From the article, 160,000 people were subjected to extra scrutiny, of these 15,000 were referred to law enforcement, of the 15,000 referred to law enforcement only 1266 were arrested.

Over all failure rate equals 99.21%, of those that the BDO deemed nefarious enough to refer to a LEO the failure rate equals 91.56%.

^^^Excellent post.

Trollkiller said 'failure rate', (which is itself a fuzzy term), not 'false positive', and your usage is not quite what "false positive" means in detection system statistics.

The "positive predictive value" of (number_of_true_positives)/(number of true_positives+number_of_false_positives) is the term of art for describing how useful the BDO's predictions are and what was quoted in the article. Trollkiller's number of 99.21 is 1-positive_predicted_value and is the percentage that you should rely on a BDO's positive report of sinister activity.

You could think of the positive_predictive_value as the rate of criminality among people who test BDO-positive. If it is significantly higher than the baseline rate of criminality, then the BDOs are doing better than chance.

"False positive" is (number_of_false_positives)/(number_of_false_positives+number_of_true_negative s). So, since 1.46Bpassengers+ were unspotted by BDOs over the 2+ years in the article, the BDO "false positive" rate is only (160,000-1,266)/1460000000=0.0001087219 or 0.01%.

Of course, these metrics completely ignore the difficult to assess "false negative" rate of how many unspotted criminals the BDOs miss. If magic BDO skill is such that they inerrantly picked out the worst 0.01% of passengers, then the false negative rate was zero.

If BDOs have no skill, (i.e, they were like a random Magic-8-ball that said "BDO-positive" 160,000/1,460,000,000=0.01% of the time) then the criminality of the BDO+ people would match the criminality of the BDO-negative people. with a false negative rate of 99.99% the leaving 1266/160,000*(1,460,000,000-160,000)=11,550,984 druggie and ID-thief passengers to be detected by other layers. That would give a BDO false negative rate of 11,550,984/(11,550,984+1266)=99.99%.

A fundamental problem with low-rate detection systems is that there isn't enough actual instances to make the performance metrics very useful. At 2,000,000 passengers per day, one terrorist in 500 days is a 1-in-a-billion rare event. It isn't a problem that visual inspection systems can handle.

(From the article, two years+ of BDOing found 160,000 BDO-positive passengers wth 1266 arrests. From elsewhere, 2,000,000 passengers per day times 2 years = 1.46Bpassengers.)

What does TSA mean when it says BDO is "effective"?

Somewhere around paragraph 3 or 4 my head 'sploded.

Maybe it either looks effective or sounds effective.... without actually being particularly effective. BDO does sound like a personal care product. Eliminate unsightly _________ with BDO! Free, at major airports, nationwide! :rolleyes:

I think it means we can terminate the BDO program and just carry around Magic 8-Balls for the same amount of effectiveness.

Oh wait, 8-Balls are not allowed since they’re filled with more than 100ml of liquid.


http://i29.photobucket.com/albums/c2...9f6apg6wm4.jpg

I'm not sure about the ARP but I was PRP and it was a glowing experience.:eek: I was in country for the Ramstein bombing and when Fredrick Kroesen's armoured car was hit by an RPG. That really added a pucker factor when flying in and out of certain "sites".

APG=Aberdeen Proving Ground, the home of the US Army Ordnance School. I guess I went to the closest ordinance school that the Army has, which is the JAG school at the University of Virginia. ;)

ordnance sounds so redneck:) I was strictly aviation and in Germany our unit transported certain ordnance

Sorry.

Wikipedia lays out a nice example here for bowel cancer, but 2 years of TSA's BDO program would have something like:

TP=1266 FP=158,000

FN=???? and TN=1,460,000,000

if you care about druggies, and

TP=<1 FP=160,000

FN=<1? TN=1,460,000,000


...if you care about terrorists.

The fractions are so close to zero or 100% that it is nearly impossible to tell if you are making things better or worse. Including druggies, ID thieves, liars, tax evaders, and speeders is what one would have to do to keep the testers from dying of boredom.

I'll 'splain. Is no time, I'll sum up:
a) 99.21% of people stopped by BDOs are not found to be doing anything wrong. (The "falsely suspected" factor.)
b) 0.79% of the people stopped by BDOs have been found to be doing something illegal. (The "sheer dumb luck" factor.)
c) 0.00000% of the people stopped by BDOs have been found to be terrorists. (The "needle-in-a-haystack" factor.)
d) 0.01% of all the people in airports were stopped by BDOs but were not doing anything illegal. (The "hassle" factor.)
e) there is no way of knowing how many people who were not stopped were doing something illegal (terrorist-related or otherwise).
f) If about 0.79% of the people who go through airports are doing something illegal, then the BDOs "secret powers" are no better than just randomly picking people out of the crowd.

But while we can play statistics all day long, I keep remembering this is an organization that agonizes over whether 3 is, or is not, 3.4, and struggles with fluid ounces versus ounces (weight). :p

When TSA says "BDOs are effective" they mean "it impresses the Kettles to have people asking questions in the airport."

I think you nailed the definition of "effective."