I really enjoyed a restaurant a short walk from Shinjuku Station called Tempura Tsunahachi and they were super friendly and I really enjoyed sitting at the counter watching the cooks prepare the tempura they would serve to me.

My tempura set was JPY2,000 but they do have different choices that can be double that amount that but I'd recommend it for good tempura in the Shinjuku area as I thought it was a really good value and would return if I was back in the area.