"People like you should not be allowed to travel"
Was he referring to people of your race? your gender? your ethnicity? your size? your religion? your....???
Probably not a good choice of words. But, unless you captured this on audio, you are, of course, lying about his comment to you.