Inland Majorca

There's a place called Soller that I like. Very low-key, a couple of good restaurants and we've stayed here a couple of times.
http://www.hotelsalvia.com/

And they don't allow kids, either! (I have three, so when me and the wife do get away for a weekend, this is quite important...)

Might fail on the cultural aspect however - the town and nearby port has an interesting history, but it's not exactly rammed with art galleries.

Do bear in mind that if you're looking for award seats, the last week of May (starting with the bank holiday weekend) is half term for UK schools.