After living near London for over a year, I've come to the conclusion that 'not terribly costly,' is hard to find but.. there are exceptions. I advise reading through the threads at www.chowhound.com (International section). There are some good ideas there plus www.london-eating.co.uk often has special deals at quite good restaurants.

If you want a nice atmosphere and Indian that's a bit different, I like The Cinnamon Club which is in Westminster. Chinatown has less expensive meals and for a quick lunch, there's always Pret a Manger which has very good sandwiches, and they are all over London. There are lots of Wagamama places if you like noodles.

Everything is relative. I really don't know what not costly means to you. Stay away from chains like Garfunkel's which are less expensive but rather tasteless. People rave about J. Sheekey for fish, Hakkasan for Chinese.. these are probably not in the less expensive range but good for a splurge.
Try a traditional pub like the Grenadier... good pub food abounds now in London and the suburbs. Belgo is a Belgium moules and chips place... also a favorite if one doesn't want a fancy meal.

Browse around the Borough Market on a Friday or Saturday.. great way to see how Londoners shop and try the Monmouth Coffee shop across the street for some delicious brew and good cake. I happen to love the large cafe at the Tate Modern - delicious food and pretty view.

I guess this could go on forever! Check the websites.. they're helpful. Good luck and enjoy your trip. It's getting chilly now - bring layers.