I tend to wear black and dark grey items from M&S's loungewear range. They are as comfortable as a tracksuit but slightly smarter (especially the long sleeve items and if you avoid the tartan look), better quality then F PJs, cool but cosy (100% cotton) and they have enough pockets etc and a reliable enough drawstring that I keep them on for the taxi trip home.
I usually change into them in the lounge before boarding.