I have been asked several times to switch seats and politely declined without any great fanfare, they just kept working the rest of the cabin until they found someone. Sometimes I have switched when it works, and sometimes not, but it's never been a major issue whenever I turn a request down.
I did have this flight from hell once flying into Cleveland on a UX flight. I get on board and sit down and there is a crew bag wedged under my seat across the entire length of the two seats in front of me on this little RJ cropduster that has no legroom anyway. Clearly labeled "crew". FA comes flying by and I explained the problem and she just ignored me and kept going forward. Me and a seatmate lugged it under the other empty seat across the aisle. We did the usual departure routine, then when we reached altitude, she just stayed forward and never served drinks or anything. It was weird, like she was having her last day of work or something.