Agtree... saw alot of families with babies and toddlers (surprised) at Andaz... the staff iwere attentive... though wonder about space situation in the actual room (layout and amenities... but we all have our opinions of this Andaz regarding form vs function. The pools and beach can be toddler friendly. It's what you want from the actual island I guess.
For older kids I think Kauai might be a better choice.
JMHO