It wouldn't seem very polite as technically speaking, an airplane seat, no matter how private is still a public space (and one that doesn't condone baring skin) and taking your clothes/top off is bound to be met with disapproval and may as well end up in one of those passenger shaming accounts. I wouldn't quite recommend doing so...

Also, some airline pyjamas are very light and quite thin, so it isn't too bad, but I always pack a light shirt and a pyjama bottom to use whenever an airline doesn't provide pyjamas or have uncomfortable ones.

Despite that, I am surprised at how relatively few people (particularly in J cabins) would change to the provided pyjamas (if any), and I'm even more surprised to see some people sleeping in a dress shirt and corresponding suit trousers which must be very uncomfortable!