Interesting question.
My guess is that TSA cannot detain you, but that the FBI or the local polcie can. I am not sure who conducts these reverse screenings, but I would guess that federal law enforcement is involved.
Refusing to submit to a search could be grounds for detention in that it arouses the suspicion of the officers doing the search. If that alone is enough grounds for an arrest, then the police can do a full search of your person and possesstions as a search incident to arrest.