The two infant boys make it harder.You might want to avoid castle hotels and cobblestone villages as well as wineries and hilltop villages.

Normally I would say train to Wiesbaden or Mainz for a relaxing and pretty small town if you don't have a car or don't want to drive.

If you're driving, the obvious thing to do would be the Romantik Road, but IMO it takes about three days to drive it in its entirely, and presumably you would need to return to Frankfurt with the car to fly home, so you might compromise and aim for the "Christmas Village" at Rotemberg-am-den Tauber (spelling?), where the Romantik Road and the Castle Road intersect. The two fantasy castles of King Ludwig are father away, closer to Fussen and Lake Constantz.

Depending on the ages, small boys might like fortresses and castles, so driving along the Rhein or going up the Mosel (Trier is nice and has a cool fortress that looks huge and serious) would be fun. Are they toddlers under age two (infants in airline-speak) or true tiny babies?