I dress casually when I fly, can't say that I've ever noticed whether I've been subject to "looks" or not, nor have I noticed what anyone else in the cabin is wearing. I think most people are more concerned with what they are doing themselves to worry about giving other passengers disapproving stares.