If I'm asked to switch and do consent, I now always ask to see that person's boarding pass. A couple of times, I've had people switch me "their" seats, which turned out not to belong to them at all. So I got to my new place and someone came and said, "Excuse me, you're in my seat." Both times, I did get my original seat back, although with a lot of commotion.
I expect visual proof now.
And I agree: poach my seat, no chance of me switching with you.