Living in the Houston area, I run into that a lot. I have a sweater designated for airline travel, and wear very thin weight pants with a short sleeved top (at my age, nobody wants to see my bare upper arms) and carry the sweater in an outside pocket of my carry-on. It comes in handy to have with me during the trip, too, because a lot of restaurants, etc. often air-condition to freezing in hot climates.