JWHITE hit the nail on the head. I've taken my assigned seat and then only after the door has closed I have felt fine to move to another seat.

The poacher in this example is a bully, you did the right thing.