I've never been asked to swap, but I usually get a window seat. I've also never asked to swap seats, but that was mainly because I either traveled alone or in very large groups, or managed to get seated by my traveling companions.

Personally, I think it's okay to ask to swap seats if it will be equal or better for the person you're asking. And if the person says no, then that should be the end of that.