I have run into this from time to time, though not entirely that frequently. I have held my elite status with CO since I was 17, flying back and forth from school, and now (I'm 24), on my own personal travels.
Every now and then, someone would mention similar comments, "This line is for FC/Elite," or "They're boarding FC right now." Never really bothered me...Although sometimes I would have a chuckle at people glancing down at my BP after making the comment about only boarding FC, as though to "check". I'd generally respond with, "Oh, great. I'm in the right place then..." or something similar.
I never really let things like that wind me up. Life's too short to concern yourself with what other people are thinking. I always viewed things like this as their issues, not mine. I knew where I belonged, and that's all that mattered.