Couple of less known tourist activities:
In Jiufen take occasion to visit the POW memorial, not many people know about the japanese labour camps at the jingguashi mine, you can see the house of the japanese commander and some other memorabilia.
in Kaohsiung, there is a very pretty Kendo dojo from the japanese time. A relic from the japanese time that is still used for teaching kendo.
in the Nantou or Miaoli area you can also visit some old paper factories where you can give it a try in paper making. If you read chinese you can find local tourist guides at Eslite bookshop with lots of interesting alternative activities.