All of downtown San Diego is walkable from the US Grant. It's a fun relaxing iconic historic property that seems IME to attract a somewhat older clientele. No pool, but the restaurant and bar are pretty good. Rooms and suites are highly variable in size, comfort, bathroom, etc. The $176 rate is excellent but you might want to consider booking a higher category room.

The historic downtown entertainment Gaslamp area has a couple upscale boutique properties that you might find appealing.

I assume that you've eliminated the W and Andaz for not being relaxing or true luxury.

GH Manchester is very good for a large convention hotel. It's in a good location and newly renovated. Lounge is good. However, I might be biased as I was given an enormous suite, the size of four rooms, for my last stay using a DSU.

It would also be fun to stay in La Jolla Village.

There's also a very nice (at the four star level, maybe even three and a half) newish Spanish style hotel with pool and spa that's very close to the entrance to the UCSD campus. It's very pleasant and relaxing, although you might want to check about group events during your dates.