All, thanks for your questions and comments. It helped us question our trip and what we want out of it. I think what we were finding is that most of the hotels were too big to be accessible to the baby by being out at the pool. Only a few islands in greece have nice hotels where the pool is right outside a poolfront room. We have been eyeing up a few interesting towns, that might give you a sense of what we want. For more clarity: We want to spend 9 nights in a smaller beach town, likely now in a nice Airbnb where we can hang out on a terrace or by a private pool while baby sleeps, and definitely close to the town center so we can walk to town with stroller/bassinet and/or baby carrier, or so one parent can head into town while the other sticks back with baby. A nice town that has a beach would be awesome, so we can walk down tot he beach when time/baby permits, or stroll a promenade for a lunch or drink if baby is sleeping. We recognize we could just be stuck in the room all day if baby isn't behaving, but at least we will be close to a town with a nice terrace view and nice place to relax. This will be a pure relax trip, not trying to do or see too much, so being close to a town, and having a really nice view from our terrace is important. A few towns that piqued our interest in early research are:

Carcoeiro - Algarve - Portugal ( I am sure there plenty of other similar towns in Algarve area)
Cifalu or San Vito Lo Copo - Sicily - Italy
Naxos/Paros - Greece
Savlletri or Polignano a Mare - Puglia - Italy

We will be coming from Paris, and headed to Rome, so direct flights both ways are important. The only one that fits the bill above is Puglia region or Sicily.

I am sure there are a ton of other quaint beach towns, so now that I have provided some more direction, I was wondering if you had any suggestions?