As someone who has a 4 year old and a new 3 month old, I’d recommend that you instead focus on destinations with direct flights which is exponentially easier with small children. St. Kitts and St. Lucia are pretty long flights.

I’m not sure if you have a direct flight to Cancun from Cleveland but if you do I highly recommend Rosewood Mayakoba; we went when our daughter was 2 as it’s perfect with little ones. They really go all out for kids and all rooms have plunge pools so you can use it when the baby is napping. The Escalade car service from the airport is well done with car seat provided. They also have an excellent babysitter service so you can go out to dinner at the hotel or another Mayakoba resort after the baby is asleep for the night. The spa is one of the best we’ve been to in our travels. We plan to go again this summer.

Harbour Island, Bahamas.
Many excellent resorts:
Pink Sands
Dunmore
Coral Sands
Ramora

I agree the Rosewood Mayakoba would be the easiest to get to (UA has many flights to Cancun). It’s a fabulous, super child friendly hotel.
Rosewood Mayakoba - Riviera Maya, Mexico

UA can get you to the equally wonderful FS in Costa Rica. But this one might be better saved for when your child is a little older and can appreciate all the animals.
TRIP REPORT: The Peace Lodge & Four Seasons Peninsula Papagayo (Costa Rica)

Both have plunge pools available in the rooms.

We went to Harbour Island this past summer (staying at a villa near Pink Sands) with our daughter and didn’t find it to be the most kid friendly island. The hotels aren’t really full resorts like we are used to with comprehensive kids programs, etc. The restaurants were geared more towards adults. I would leave that hotel until the child is older.

Agreed on FS Costa Rica— we were there when my daughter was 3 in December 2017. The only bummer there is that the pools aren’t heated and they were cold. Our suite’s plunge pool was plenty warm but the hotel pools were too chilly for my daughter, including the kids club pool. I thought it was strange they didn’t heat at least that pool. I think that’s a perfect hotel for 6 year olds plus bc they’re old enough for the kids club and some of the volcano and jungle day trips which are too long for a smaller child.

When we stayed at the Rosewood we ate at the Thai restaurant at Banyan Tree (very good) and also toured a villa there—the Rosewood IMHO is much nicer and more upscale. Better maintained and judging by the reviews, better service. The Rosewood butler system actually works well which is not the case at most places we’ve been.. There is a new adult only beachfront pool and lounge section at Rosewood that looks nice that one of you can use when the baby is napping.

For room types at Rosewood, the ocean view pool suite worked really well with our 2 year old because it had sliding doors between the living room and entrance area, and between the entrance and bedroom (the floor plans are all online). We put the hotel provided crib in the living room, closed the sliding doors that separated it from the entrance door such that we could get room service at night without disturbing her. Very helpful because the hotel babysitter could come at night after she was asleep and closed off in living room. Really good setup. There is a nice rooftop plunge pool (large) with great water view. With a small baby and likely a 3 nap a day schedule I’d recommend you get a room on the beach side as opposed to lagoon side so you can take turns going to beach etc. quickly while baby is napping. That suite or the beachside lagoon view suite would be my category recommendation.

That short (for going to Cancun) nonstop is alluring, I agree. Cabo is not our favorite Mexican locale (that would be Zihuatanejo or Costalegre south of Puerto Vallata) but we are there often because it's a mere two-hour and 40-minute nonstop from SFO.

The Maroma/Playa del Carmen area, 40 minutes south of Cancun, has wonderful Caribbean beaches. With babe in tow, I recommend Belmond Maroma, most authentic of the best resorts in the area. The beach there is close to the suites, powder soft sand, calm, little surf, warm waters and turquoise beautiful. Beach toys and floats provided. It is manicured and cleaned daily. The suites are lovely, some with large plunge pools.

Food is fine enough if you're not gourmands (we are not). Beach service is super fab and they are very kind to kids and accommodating of same. Beautiful at night when hundreds of lanterns are lit throughout the property.

The other resort I recommend is in Playa del Carmen, directly on the beach -- Mahekal Beach Resort. It's not luxury, more like three-star but it's great for families with babes and kids. The venue could not be more accommodating -- kids are positively revered. The beach is excellent and just steps away. There are a variety of palapa accommodations including three large stand-alone beachfront rooms with small private pool, spacious porch with hammocks, etc. Two meals daily included. Breakfast is wonderful, lunch quite good, dinner decent. But it's Playa del Carmen so many lovely places to dine. Cabs helaciously inexpensive. Very convivial place. Lovely staff and service.
https://www.mahekalbeachresort.com/.

Agree with Kat that the Belmond Maroma is another nice option. Most beautiful beaches, authentic Mexican, very quiet. It’s not as outwardly child friendly (no kids club or dedicated amenities) but the service is so good they will take wonderful care of your baby.