Originally Posted by
JSmith1969
Bart, there was a link in the post Trolkiller was quoting, as was clearly apparent to anyone who can read. According to the figures in that link, 160,000 people have been harassed by BDOs, and only 1,266 arrests have been made as a result of this harassment, and vast majority of those were for things like fake IDs and drugs, none of which have anything to do with security. 1,266 divided by 160,000 is 0.0079125, which means that Trollkiller is right to say that the BDO has a false positive rate of over 99% and is thus prima facie useless and does nothing to make anyone safer.
Trollkiller said 'failure rate', (which is itself a fuzzy term), not 'false positive', and your usage is not quite what "false positive" means in detection system statistics.
The "positive predictive value" of (number_of_true_positives)/(number of true_positives+number_of_false_positives) is the term of art for describing how useful the BDO's predictions are and what was quoted in
the article. Trollkiller's number of 99.21 is 1-positive_predicted_value and is the percentage that you should rely on a BDO's positive report of sinister activity.
You could think of the positive_predictive_value as the rate of criminality among people who test BDO-positive. If it is significantly higher than the baseline rate of criminality, then the BDOs are doing better than chance.
"False positive" is (number_of_false_positives)/(number_of_false_positives+number_of_true_negative s). So, since 1.46Bpassengers+ were unspotted by BDOs over the 2+ years in the article, the BDO "false positive" rate is only (160,000-1,266)/1460000000=0.0001087219 or 0.01%.
Of course, these metrics completely ignore the difficult to assess "false negative" rate of how many unspotted criminals the BDOs miss. If magic BDO skill is such that they inerrantly picked out the worst 0.01% of passengers, then the false negative rate was zero.
If BDOs have no skill, (i.e, they were like a random Magic-8-ball that said "BDO-positive" 160,000/1,460,000,000=0.01% of the time) then the criminality of the BDO+ people would match the criminality of the BDO-negative people. with a false negative rate of 99.99% the leaving 1266/160,000*(1,460,000,000-160,000)=11,550,984 druggie and ID-thief passengers to be detected by other layers. That would give a BDO false negative rate of 11,550,984/(11,550,984+1266)=99.99%.
A fundamental problem with low-rate detection systems is that there isn't enough actual instances to make the performance metrics very useful. At 2,000,000 passengers per day, one terrorist in 500 days is a 1-in-a-billion rare event. It isn't a problem that
visual inspection systems can handle.
(From
the article, two years+ of BDOing found 160,000 BDO-positive passengers wth 1266 arrests. From elsewhere, 2,000,000 passengers per day times 2 years = 1.46Bpassengers.)
What does TSA mean when it says BDO is "effective"?