Honestly I will not head out because Taipei is a "late rise" city. So most restaurants and shops will not be opened at those early hours.
If it is 4pm to 10pm, it will be different because you can take MRT and go to the outlet mall in Taoyuan, which are just a few stops away from the airport stop.
Unfortunately those early hours mean that spending some times in the terminal will be best.

EVA Air lounge is not the fanciest, but at least you can take a shower and have a space to rest. The lounge will be busy for the first half of your stay, but will quiet down after the morning bank flights have departed.

While Taoyuan airport is no Singapore Changi, it is still okay. Do some walking along the B/C corridor will be good too, after a long flight.

Carfield