For anyone who doesn't have enough time to leave the area, but wants to hang out away from the terminal, there is an aviation museum (although somewhat outdated) on the TPE airport campus. There is also an observation tower to watch take offs/landings and hear ATC communication.

Both of these are right next to the Novotel, and I believe there is a free shuttle bus to/from the terminal.