Yes you can but you need to go through Immigration/Custom formalities. If your a citizen of a Cputry that needs a Visa to enter Taiwan, you will be required to have one before your arrival.

FYI, not much around the airport, maybe a mall or two. Depending on traffic, it might take you 45-1hr to get tp TPE City.