Some airlines seem to be printing some indication on the boarding pass that you've been selected - then the gate agt just has to look for the indicator, not a list of seat numbers.

Others (I've seen Delta do this) have the gate agt look you in the face and determine at the time of boarding if your are male traveling alone or non-native US - er - I mean if you are randomly selected.

But if there's no indication on the pass and they're just calling seat numbers, well, I, too, wondered if you could skip the process, but I'm probably too timid to test it.