One other issue is, how much do you care about the character of the town itself, as opposed to the resort?

I would say that I prefer Puerto Vallarta, then San Jose del Cabo, then Cabo San Lucas, in that order, with CSL a distant third, having much less character than the other two. Another point is that Punta Mita is not that near to Puerto Vallarta proper, whereas Palmilla is pretty close to San Jose del Cabo.

Personally, I like the beach by Palmilla, it's one of the only swimable beaches on the Cabo Corridor. I also like that there is an extensive fleet of reasonably priced fishing pangas that leave right out of Palmilla.