Definitely go to Shilin Night Market. It's a (literal) feast for the senses and absolutely wonderful, not just for the food but also for the people-watching. When I was in Taiwan a few years ago, I went on my last night in the country and actually had thought about skipping it because I had been to something like 10 night markets already over the course of the trip and thought, how much different can it be? Well, I was very glad I went. Yes, it's busy and touristy, but the sheer size of it, the breadth of the food and shopping opportunities, the crowds, the lights, the colors... it was a great time. Would I want to go there all the time if I lived in Taipei? Probably not. But if you're just there for a few days, I think it's well worth a visit for a couple of hours.

Dalong St. night market near the temples is also good, albeit much smaller. And I second the recommendation for Ningxia night market, which was right by my hotel - that was a good one too. The smaller markets tend to be more geared to local residents, I think.

I don't know about a local city museum, although I can say that I loved the National Palace Museum and went there twice when I was in Taipei.

One other recommendation I have is to take the Maokong Gondola cable car up to the top of the mountain and have tea or lunch at one of the tea houses, then walk some of the trails and check out the big temple. That was a highlight for me.

As to ATMs, I'm from the US, but I can say that I never had a problem using my debit card anywhere in Taiwan, including at non-bank ATMs with weird names at 7/11 and the like. I'd just keep an eye on fees, as I recall they can differ a great deal from machine to machine.