I think that visits to any of the places you’ve mentioned can be tailored to your needs. There are certain things which you can hire at your destination if need be (e.g. a stroller which you’ll need for only a day or two). I think the key is never plan to do too much. Whether it’s driving, hiking, sightseeing, etc. always plan for a bit less than you would ideally like to complete. That doesn’t mean cutting out something completely, but just prioritize certain things where possible.

I would definitely agree with CDTraveler that a more outdoor-focused trip will have some unique aspects you’ll need to think about, and that each trip will be different from the last. For example, we made a mistake when we first traveled with our daughter when she was a baby; we took her stroller to use not thinking that it would be impractical on cobbled pavement, and places which only had stairs or steps. It seems simple, and we learned quickly, but with each trip we always learned more about what worked and that made the next trip easier. Some of those things we never did twice, others we still use/do even now.