what you described is what i would always tell my staff to do at the bank when verifying i/d. look at the i/d, look at the name, look up at the face, then say "thank you <name>" when about to hand the i/d back. it's a simple way to determine if the person is in fact the person named on the i/d as if there is something phunky, a) the person will not make eye contact with you and b) they won't acknowledge the name on the i/d when you thank them.
now, what you have described has happened to me at sfo, iad and bos and i don't pull any punches as when it does happen, i go thru the line and ask for a supervisor after i'm screened. i'd do it sooner but i don't want to hold up the line and i'll say this-sfo & bos were none too happy but iad couldn't give a rat's a$$.........