I'd recommend a great place called Cru. It's a wine bar with a nice prix fixe. Not open for lunch though. Not exactly 'downtown' but around where some of the previous recommendations are...

You'll love Vancouver. It's a beautiful place.