I'm a share-and-share-alike kind of guy. If you rest only your elbow (rather than your entire arm) on the armrest, you and your seat opponent can both enjoy two armrests. It's just a matter of occupying the smallest amount of the armrest to make you comfortable.

I'm not sure what Emily Post would have to say...