I agree 100% with those who suggest you mention your propensity for motion sickness. It's well known among pilots that some individuals are calmed by by looking towards the horizon rather than staring at static instrument panel in front of them.
Additionally, many pilots get airsick unless they are actually the pilot in command, so different strokes, eh?! Your case is seems a little different - but whatever works.
Without knowing the whole story behind your phobia, your seatmate doesn't have the opportunity to be compassionate as he most likely would mistake your actions as being based in either indifference or just plain selfish behavior.