I feel your pain. I travel almost exclusively with a backpack. I stay in $10 a night backpacker hostels, but sometimes I splurge (or cash in points) and stay some place nice. I also fly mostly J when traveling internationally.
Personally, I get a kick out of it. There's nothing you can do about it. People are going to judge you. I try and be as polite as possible and let people make idiots of themselves when they make incorrect judgments.
A Delta lady physically blocked me from entering the BusinessElite checkin line and told me "everyone has to use the Kiosk". She never asked me if I was flying B/E, she just assumed I wasn't. When I politely insisted that I'd rather use the line she demanded my boarding pass and begrudgingly let me proceed. I think my smirk communicated more than any amount of words could have
