Not at all, common practice for many, including myself.
Change at the lounge pre-flight and then back into your other clothes shortly before landing or in an airport bathroom. Depending on your destination, these will be a bit more comfortable than the plane (or pretty nasty).