I have to admit, I'm one of those people that likes to dress up a bit when traveling, whether in coach, business or first. For a very long time I wasn't able to travel, so it's still fun, and a special experience for me.
I guess I shouldn't care what anyone else wears, but I do get surprised at the variety of ways people dress on planes. I can understand why a flight attendant would think someone was not in first class, but as you've all said, you can't tell any more who flys in what cabin based on how they're dressed.
I say you should be comfortable when you fly, but if you are putting your flip flops on the back of my armrest, then I might have a problem. I personally wouldn't care if you're sitting next to me in shorts and a t-shirt as long as I don't see anything (if you know what I mean) and you're not reeking of BO. We all know how tough it is to stay fresh when traveling but I try to make sure I smell decent even if I don't look that great.
I'm on a college campus right now, so I'm constantly reminded how the younger set dresses these days, lots of underwear showing and sometimes more (from both men and women). If I have to see your "anatomy" on a plane, I won't be happy.