Go East - hire a driver

Taipei - Zoo for the pandas and maokong gondola, daytrip to jiufen
Silks Yilan - go visit a farm or the brick art museum. otherwise the hotel is very kid friendly and also hot springs
If you like hiking (both easy and hard) or just beautiful scenery goto toroko gorge. Silks is the best hotel in the park.
If you want to sleep in an aquarium and fall asleep or wake up to dolphins, rays, etc Hualien ocean park
Liyu Lake is very pretty. Good for biking, hiking, boating.
Great cycling (including family friendly e-biking) in Taitung and just marvel at the pacific ocean on one side and mountains on the other. The whole East coast is a beautiful drive.
Relax at a hot springs resort at hotel royal chihpin
Fly back to taipei or take the train to save time from Taitung.

Or go West
Taipei
High speed rail and hotel shuttle to Sun moon lake
Alishan - a bit of trek to get to but beautiful, especially sunrises and if you want to see a sea of clouds from the mountain top. Also if you like tea. You can get a driver to go from sun moon lake to alishan, but beware it's winding mountain roads the whole way.
Tainan - old capital and food haven, but you need to taxi. Public transport is not great.
Kaohsiung - Pier 2 is nice, but if you don't want another city it's passable.
Pick either kenting or Xiaoliuqiu. Kenting has bigger hotels like Ceasars park. There's also an aquarium where you can sleep in the same room with a beluga whale. Xiaoliuqiu is Airbnb only, but you can goto the beach and swim with turtles. Neither beach will compare to aussie beaches

If you want less traveling within Taiwan, you can stick to the north.
Taipei
Silks Yilan
Westin Dashee in Taoyuan 40 mins from airport. No beach, but nice pool and lots of kid activities. There's window on China theme park which is a similar to a miniland at legoland plus kiddie rides. There's also yingge if you like ceramics (tho probably not infant friendly) and shimen reservoir which is pretty and good for walking or hiking.

Tons of hiking, biking, and outdoorsy activities everywhere in Taiwan. Less water activities, especially compared to Australia.