hotspring in Taitung is located mainly in Chihpen, the Royal Chihpen hotel is a good choice although a bit dated.
There is a hotspring in Taroko but it has been closed for a couple of years now after a fatal accident in which a tourist was killed by falling rocks.
The nearest hotspring from Hualien would be in Ruishui, which is about 1.5 hr's drive from town.
Both Taroko and Alishan are accessible now.
If you choose to visit Alishan and wants to stay near the forest recreation area, Alishan Guest House would be your best bet.