Not sure if you would still consider it "downtown", but I find that Bloor Street around Bathurst has a lot of nice, inexpensive Asian places (Korean, mostly). They feel authentic to this Westerner...
And, somewhat counterintuitively: The Momofuku noodle bar at the Shangri-La (University & Richmond) makes a wicked bowl of Ramen ($20 after tip + tax).