I was in Morocco in 2002. So I imagine a fair bit has changed since then. But at that time Essaouira was one of my favourite places I visited in the whole country. Beautiful seaside town with a nice relaxed feel.
Didn't go surfing or horseback riding but I do remember that there was quite a wind on some days.