You were flying out of Japan.

I'd bet everything I own that the flyer was Japanese and was assigned that seat. The thought of moving never crossed their mind. It's a cultural thing (my wife is Japanese). The combination of following rules and living in an area where there's a lack of personal space means the entire plane could have been empty and they still would have sat there.