For lunch, my hubby and I like Brandy Ho's. Hot, hot, hot! ^
Based on recommendations here, we ate at Zuni and Boulevard for dinner. Zuni is awesome. We can't wait to go back there. Boulevard also has great food, but we found the service was so technically proficient it was cold. Don't even try to joke around with the staff.

I know you said not the usual tourist spots, but consider Alcatraz at night if you haven't done it. We waited until almost closing when everyone else had left and got some great black and white shots of the place with no one in them.