On an airplane, I've always practiced the rule that the aisle and window persons should allow the middle person both arm rests as the sole consolation of having that horrible seat.
At a performance, I think it is first come-first served with regards to arm rests.