I've traveled quite a bit in both with small children, so I have some experience. Both are very well set up for travel with children. Clean public bathrooms with changing tables are plentiful, and you can take kids pretty much anywhere (e.g. restaurants).

Taiwan being further south means that you have a better chance to see warm weather. Kenting is a great choice as it's far south and so sees warm weather nearly year-round. Beaches are ok, not the best but not bad, and there are a few resorts. I would agree that Taiwan's smaller size makes sightseeing a bit more manageable. I would disagree that Taiwan is boring or expensive. That's somewhat subjective but I find it to be neither. One downside of Taiwan is that sidewalks in cities tend to be uneven and can be taken up by shop/restaurant seating or motorbike parking. This makes pushing a stroller around hard. Taipei is a great city with plenty to do. There's a hot springs resort in Beitou north of the city, accessible by subway. Beitou and Yangmingshan, Elephant Mountain (Xiangshan), and the Taipei Zoo plus Maokong Gondola make for great day walks/hikes.

If you are aiming for April/May, most of Japan from Tokyo on south will be warm by then. Okinawa has plenty of beach resorts, as do further out islands like Miyakojima and Ishigaki and some of the beaches are really top notch. The big cities are not likely to be quiet or relaxing vacations but there's plenty to see. Tokyo has the Railway Museum which is good for half a day. Lots of kids love trains and just riding them around to your next destination is fun enough. Restaurants in Japan are very likely to have high chairs and a set of kids' plate and utensils. The yen rate is very favorable to many currencies now, so visiting Japan is much more affordable than in the past, but it's not vastly cheaper than Taiwan.

I'd say pick one for now and save the other for when the kids are older. You won't likely go wrong with either one.