that's not much time. And depending when you plan to visit the weather could slow you down.

CKS Memorial hall is close to the main station - just 2 stops on the MRT away. Not sure it's really something a toddler would enjoy but the history was interesting and it's fairly impressive

The zoo might be a good option. I'm not a huge fan of zoos in general but I thought Taipei zoo was fairly nice. Or the Maokong cable car is also right there next to the zoo. Takes you up the mountain to tea houses and plantations. It's nice area to walk around.

There are a number of temples around town. The Confucius temple and Baoan temple are right next to each other. They're nice in the evening when they're lit up.