Something similar happened to me a few years back. I had booked a window seat, and when I got to my seat, a mother and her child were in the window and middle. The mother "asked" me if I would change with her (her seat was a middle a row back).
Seat poachers get an automatic, and emphatic "No".