Depending on what age baby will be when you go, perhaps look for a hotel with a nice park nearby? Tokyo accommodation with Toddlers

I'd look at room size when comparing various options, too. While you could get a good cash deal on 18-22 sq meter JP business hotel room for all three of you, it might be a bit snug for baby gear, longer trip amounts of luggage, and a crib if you use one. If you are going in hot season and aren't from a similar climate, then I'd probably look for a hotel closest to what your spouse likes to do in (her former) spare time to reduce time out in the heat. There's some decently well regarded western hotels which are kind further walking distance from a well connected subway station (with escalator or elevator) or surrounded by offices rather than cute cafes. Or uphill when returning from a JR station (though offering to push the stroller instead of spouse could work around that) or accessibility inconvenience due to stair only pedestrian overpasses. Google Maps street view would give you a decent idea.

LapLap also has good posts about what is easier in Japan with kids, including looking for eateries with tatami mats and what's more accessible with strollers https://www.flyertalk.com/forum/35010311-post9.html