For me it all boils down to the common courtesy of asking. I’ve had people take my seat and just assume I’d be happy to move to accommodate their wants / needs and I’ve told them in no uncertain terms that I’m not swapping seats.

I have swapped where it’s the decent thing to do (for example, a father flying with their young daughter where she was clearly apprehensive about being separated) but I have a zero tolerance policy where people just assume I’ll swap because I’m a solo traveller.