Barnsdale Lodge, a hotel and restaurant is just outside of Oakham might be worth a look. I went there for lunch a few times.

I also remember the whipper inn to be good, but yes it is right next to the market square.