You did the right thing. No need to make yourself less comfortable. And while I love my wife, if we were separated on a 5 hour flight, I could live with it. It's not as if I don't get a chance to see her otherwise!