While it is no fun having one's knees in the seat in front of them, it is also no fun having to cross your arms, either. Being broad-shouldered, and a polite person, that's what I have to do when the row is full. Makes eating really amusing.
Since I am always in a bulkhead or Exit Row and GAs fill front to back so there will be rows with empty middles even when mine isn't and most folks take the Exit Row, even if they have to sit in the middle, I am almost always in a full row, so I can count on one hand the number of times I have had an empty seat next to me. But it sure makes a difference in terms of comfort (especially since I usually try and lean out into the aisle a bit to give the middle person some extra shoulder room, which results in me getting tagged by everyone and everything passing by

).