I would be a quiescently-frozen Personsicle if I wore shorts on a plane. But, if it's comfortable for you, do it.

I always wear dress pants, comfy-but-dressy shoes, a shirt/blouse and some kind of outerwear depending on the weather both here and there. (The first three are thanks to my mother's voice in my head.)

The garment bag is always with me, if a change is required. I usually have something dressier and something more casual in there, along with a few things that make getting stranded less unpleasant.