Latest GAO report on SPOT program
#1
Original Poster
Join Date: Aug 2006
Location: NW Fla. - VPS, PNS
Programs: DL, NW, HH
Posts: 333
Latest GAO report on SPOT program
I'm reading the GAO report titled "TSA Is Taking Steps to Validate the Science Underlying Its Passenger Behavior Detection Program, but Efforts May Not Be Comprehensive." It's fascinating. http://www.gao.gov/products/GAO-11-461T
My favorite little gems (so far):
As GAO reported in May 2010, TSA deployed its behavior detection program nationwide before first determining whether there was a scientifically valid basis for the program. According to TSA, the program was deployed before a scientific validation of the program was completed in response to the need to address potential security threats. However, a scientific consensus does not exist on whether behavior detection principles can be reliably used for counterterrorism purposes, according to a 2008 report of the National Research Council of the National Academy of Sciences.
-----
As we reported in May 2010, an independent panel of experts could help DHS develop a comprehensive methodology to determine if the SPOT program is based on valid scientific principles that can be effectively applied in an airport environment for counterterrorism purposes. Thus, we recommended that the Secretary of Homeland Security convene an independent panel of experts to review the methodology of the validation study on the SPOT program being conducted to determine whether the study’s methodology is sufficiently comprehensive to validate the SPOT program.
-----
Because such questions exist, the results of an independent panel of experts to assess the methodology of the study could provide DHS with additional assurance regarding whether the study’s methodology is sufficiently comprehensive to validate the SPOT program.
-----
Moreover, DHS stated that its current effort to validate the science underlying SPOT includes 3 years of operational SPOT referral data and preliminary results indicate that it is supportive of SPOT. However, in May 2010, we reported weaknesses in TSA’s process for maintaining operational data from the SPOT program database. Because of these data-related issues, we reported that meaningful analyses could not be conducted to determine if there is an association between certain behaviors and the likelihood that a person displaying certain behaviors would be referred to a law enforcement officer or whether any behavior or combination of behaviors could be used to distinguish deceptive from nondeceptive individuals.
-----
Using CBP and Department of Justice information, we examined the travel of key individuals allegedly involved in six terrorist plots that have been uncovered by law enforcement agencies. We determined that at least 16 of the individuals allegedly involved in these plots moved through 8 different airports where the SPOT program had been implemented. Six of the 8 airports were among the 10 highest-risk airports, as rated by TSA in its Current Airport Threat Assessment. In total, these individuals moved through SPOT airports on at least 23 different occasions. For example, according to Department of Justice documents, in December 2007 an individual who later pleaded guilty to providing material support to Somali terrorists boarded a plane at the Minneapolis-Saint Paul International Airport en route to Somalia. Similarly, in August 2008, an individual who later pleaded guilty to providing material support to al Qaeda boarded a plane at Newark Liberty International Airport en route to Pakistan to receive terrorist training to support his efforts to attack the New York subway system.
-----
I recommend you read the full report. It's fascinating. Basically, there's SPOT was deployed with no scientific evidence that it works and TSA & DHS have not gathered the empirical data necessary to prove that it does or does not work.
My favorite little gems (so far):
As GAO reported in May 2010, TSA deployed its behavior detection program nationwide before first determining whether there was a scientifically valid basis for the program. According to TSA, the program was deployed before a scientific validation of the program was completed in response to the need to address potential security threats. However, a scientific consensus does not exist on whether behavior detection principles can be reliably used for counterterrorism purposes, according to a 2008 report of the National Research Council of the National Academy of Sciences.
-----
As we reported in May 2010, an independent panel of experts could help DHS develop a comprehensive methodology to determine if the SPOT program is based on valid scientific principles that can be effectively applied in an airport environment for counterterrorism purposes. Thus, we recommended that the Secretary of Homeland Security convene an independent panel of experts to review the methodology of the validation study on the SPOT program being conducted to determine whether the study’s methodology is sufficiently comprehensive to validate the SPOT program.
-----
Because such questions exist, the results of an independent panel of experts to assess the methodology of the study could provide DHS with additional assurance regarding whether the study’s methodology is sufficiently comprehensive to validate the SPOT program.
-----
Moreover, DHS stated that its current effort to validate the science underlying SPOT includes 3 years of operational SPOT referral data and preliminary results indicate that it is supportive of SPOT. However, in May 2010, we reported weaknesses in TSA’s process for maintaining operational data from the SPOT program database. Because of these data-related issues, we reported that meaningful analyses could not be conducted to determine if there is an association between certain behaviors and the likelihood that a person displaying certain behaviors would be referred to a law enforcement officer or whether any behavior or combination of behaviors could be used to distinguish deceptive from nondeceptive individuals.
-----
Using CBP and Department of Justice information, we examined the travel of key individuals allegedly involved in six terrorist plots that have been uncovered by law enforcement agencies. We determined that at least 16 of the individuals allegedly involved in these plots moved through 8 different airports where the SPOT program had been implemented. Six of the 8 airports were among the 10 highest-risk airports, as rated by TSA in its Current Airport Threat Assessment. In total, these individuals moved through SPOT airports on at least 23 different occasions. For example, according to Department of Justice documents, in December 2007 an individual who later pleaded guilty to providing material support to Somali terrorists boarded a plane at the Minneapolis-Saint Paul International Airport en route to Somalia. Similarly, in August 2008, an individual who later pleaded guilty to providing material support to al Qaeda boarded a plane at Newark Liberty International Airport en route to Pakistan to receive terrorist training to support his efforts to attack the New York subway system.
-----
I recommend you read the full report. It's fascinating. Basically, there's SPOT was deployed with no scientific evidence that it works and TSA & DHS have not gathered the empirical data necessary to prove that it does or does not work.
#2
Join Date: Dec 2008
Posts: 1,444
#3
Join Date: Oct 2010
Location: Denton County, TX
Programs: AA Executive Platinum, Hilton Diamond
Posts: 737
I read the entire article. Although it sounds somewhat critical of the program and the newer technologies such as "FAST", I am very concerned about the fact that these technologies and practices are even being considered in the first place.
The main issue I have is that, like scanners, I do not believe it is appropriate to use dragnet enforcement techniques that subject people who have not given probable cause for any wrongdoing, to these technologies. I vehemently oppose walking through a portal which is going to detect my vital signs, facial expressions, body sweat, etc... My body is my personal property and what I am experiencing at any given moment of the day is NOT anyone's business.
Furthermore, how can anyone even begin to make such a leap as to propose that just because someone's heart rate is elevated, or they are sweating means that they are up to something malicious? We are all the subjects of our emotions every minute of the day and these emotions are the result of our current experiences: worries about missing our flight, personal concerns, work concerns, anger at something that happened earlier in the day, etc.... The idea that I may be stopped and questioned by some high school drop-out who thinks I'm suspicious because I'm sweating is absolutely horrifying, dispicable, and unacceptable. We are under no obligation to speak to people anyway. So if we are stopped because of a biometric anomaly and refuse to speak to anyone they are going to try to deny us access to our flight??
Complete and total bull****. I will fight this to hell and back if anything even remotely related to what is described in this article is attempted to be implemented in this country.
The main issue I have is that, like scanners, I do not believe it is appropriate to use dragnet enforcement techniques that subject people who have not given probable cause for any wrongdoing, to these technologies. I vehemently oppose walking through a portal which is going to detect my vital signs, facial expressions, body sweat, etc... My body is my personal property and what I am experiencing at any given moment of the day is NOT anyone's business.
Furthermore, how can anyone even begin to make such a leap as to propose that just because someone's heart rate is elevated, or they are sweating means that they are up to something malicious? We are all the subjects of our emotions every minute of the day and these emotions are the result of our current experiences: worries about missing our flight, personal concerns, work concerns, anger at something that happened earlier in the day, etc.... The idea that I may be stopped and questioned by some high school drop-out who thinks I'm suspicious because I'm sweating is absolutely horrifying, dispicable, and unacceptable. We are under no obligation to speak to people anyway. So if we are stopped because of a biometric anomaly and refuse to speak to anyone they are going to try to deny us access to our flight??
Complete and total bull****. I will fight this to hell and back if anything even remotely related to what is described in this article is attempted to be implemented in this country.
#4
Original Poster
Join Date: Aug 2006
Location: NW Fla. - VPS, PNS
Programs: DL, NW, HH
Posts: 333
Sadly, we have to fight within the system we have. The GAO points out the issues (including lack of empirical data that SPOT works), it is up to us to push the issue.
Instead of saying "I don't think SPOT works" you can say "The GAO reported 'key individuals allegedly involved in six terrorist plots that have been uncovered by law enforcement agencies. ...at least 16 of the individuals allegedly involved in these plots moved through 8 different airports where the SPOT program had been implemented. Six of the 8 airports were among the 10 highest-risk airports, as rated by TSA in its Current Airport Threat Assessment. In total, these individuals moved through SPOT airports on at least 23 different occasions.'"
This is empirical evidence supporting the ineffectiveness of the SPOT program, not an opinion.
Instead of saying "I don't think SPOT works" you can say "The GAO reported 'key individuals allegedly involved in six terrorist plots that have been uncovered by law enforcement agencies. ...at least 16 of the individuals allegedly involved in these plots moved through 8 different airports where the SPOT program had been implemented. Six of the 8 airports were among the 10 highest-risk airports, as rated by TSA in its Current Airport Threat Assessment. In total, these individuals moved through SPOT airports on at least 23 different occasions.'"
This is empirical evidence supporting the ineffectiveness of the SPOT program, not an opinion.
#5
FlyerTalk Evangelist


Join Date: Jul 2005
Location: The Sunshine State
Programs: Deltaworst Peon Level, TSA "Layer 21 Club", NW WP RIP
Posts: 11,372
While you have a room full of experts. . .
Thus, we recommended that the Secretary of Homeland Security convene an independent panel of experts to review the methodology of the validation study on the SPOT program
add From the end of the summary: "GAO provided the updated information to TSA. TSA had no comment."
Last edited by Flaflyer; May 11, 2011 at 9:51 am Reason: add
#6
Moderator: Smoking Lounge; FlyerTalk Evangelist



Join Date: Feb 2004
Location: SFO
Programs: Lifetime (for now) Gold MM, HH Gold, Giving Tootsie Pops to UA employees, & a retired hockey goalie
Posts: 29,078
<cue Three Stooges theme/three blind mice>
See spot run
See spot run
They haven't caught a terrorist
They haven't caught a terrorist
Oh, see spot run
See spot run
See spot run
They haven't caught a terrorist
They haven't caught a terrorist
Oh, see spot run
#7
A FlyerTalk Posting Legend




Join Date: Jun 2005
Posts: 72,273
For me, the entire report can be distilled down to these three statements:
According to TSA, the program was deployed before a scientific validation of the program was completed in response to the need to address potential security threats.
However, a scientific consensus does not exist on whether behavior detection principles can be reliably used for counterterrorism purposes, according to a 2008 report of the National Research Council of the National Academy of Sciences.
Because of these data-related issues, we reported that meaningful analyses could not be conducted to determine if there is an association between certain behaviors and the likelihood that a person displaying certain behaviors would be referred to a law enforcement officer or whether any behavior or combination of behaviors could be used to distinguish deceptive from nondeceptive individuals.
Or, to distill it even further - Waste, Fraud, and Abuse.
According to TSA, the program was deployed before a scientific validation of the program was completed in response to the need to address potential security threats.
However, a scientific consensus does not exist on whether behavior detection principles can be reliably used for counterterrorism purposes, according to a 2008 report of the National Research Council of the National Academy of Sciences.
Because of these data-related issues, we reported that meaningful analyses could not be conducted to determine if there is an association between certain behaviors and the likelihood that a person displaying certain behaviors would be referred to a law enforcement officer or whether any behavior or combination of behaviors could be used to distinguish deceptive from nondeceptive individuals.
Or, to distill it even further - Waste, Fraud, and Abuse.
#9
Join Date: Dec 2010
Location: Monterey Bay Area
Programs: Independent Libertarian
Posts: 326
BDO, SPOT, Lie Detectors.
IF these programs are effective it would ONLY be with small time non-pros.
They would be of little use in dealing with a professional who would have had the training needed to defeat them.
I would imagine the methods are taught at Langley.

