Swiss F Pajamas can look fine under a nice v-neck sweater, but, in general, I recommend wearing real clothes. I confess I wore such an outfit (accompanied by jeans, socks, and shoes in case anyone was getting sicko ideas) to run down from my apartment to buy cigarettes and club soda the other morning.
The onemileatatime blog had a rather amusing tale of the faux pas of wearing airline pajamas into a lounge.