Okinawa and Ishigaki have been on my list for a short hop from TPE.

If you're willing to leave for a weekend I really think you can bump the flight time up to 3.5hrs or so, which gives you a lot more options. I'd also focus on cities that have nonstop routes from TPE/TSA but might not be that easy to get to even if you visit the country in question later - e.g., Japanese cities like Fukuoka or Sapporo, or even secondary cities like Takamatsu, Toyama, or Sendai. Same with Korea and cities like Jeju.

If you're not totally set on leaving Taiwan, also consider other Taiwan destinations, like Kinmen/Penghu. Don't forget any of the cities the HSR stops in either.