I'll start with 2 things. First, what does '3 days' actually mean? If you arrive on a Monday night and then spend Tuesday, Wednesday, Thursday in Vancouver and then leave on Friday, that to me is 3 days. I only count full days spent in a place. Your arrival day and departure day are very rarely actual full days spent in a place.

With even 3 full days, I would not suggest spending a day going to/from Victoria. You'll spend half the day just getting to from, even if you go by plane. I don't consider that as best use of time. Time spent actually moving between places is lost time in my view. Vancouver itself has more than enough to keep the average visitor busy for 3 full days. Unfortunately, your timing is not that great for 'outdoor' activities other than skiiing at nearby Whistler Mountain. But then, I'm not sure why someone would have 'outdoor' and 'city' in the same sentence unless you are only referring to walking rather than driving.

Second, expensive is a word I dislike seeing in responses. Expensive is defined only by the individual, not by what someone else thinks is expensive or not. Vancouver hotels are no more expensive than hotels in many cities in N. America and the world.

Now that you have indicated your preferred budget for a hotel room, you have been given some good examples to work with. I happen to like the Hotel Vancouver which is arguably the most historic hotel in the city. Given your timing and criteria, I would say a downtown hotel for sure. Vancouver is actually a very walkable downtown city core. Just make sure you bring a large umbrella.