You can't go wrong either way. One of the big draws for families at the Ritz is their impressive Ritz Kids program, so there will be more families there than you might otherwise expect. But the weather in Wailea is generally even more perfect than up in Kapalua, so if you choose to go down there my favorite resort is the Wailea Beach Resort Marriott, which has a spectacular adults-only ocean-facing infinity-edge pool. While there may be kids out and about (just like anywhere in Hawaii), that is a secluded paradise.