It's possible that you might find Tokyo less overwhelming that other big cities that you're used to - Some neighborhoods are built on a human scale. The streets are less frantic and chaotic than other big cities that I can think of. You rarely hear horns and sirens in the way that you do, when wandering around large US cities.... But YMMV.
Kyoto seems like an obvious "smaller city" to use as a base. In terms of touristic sites and history, it has plenty to offer in its own right. It's also conveniently located for daytrips to Nara, Koya, Osaka, etc. If you can fly into KIX, that's the closest airport. From NRT, you need to travel 3.5 hours by train. (Or you could connect to a flight to ITM, but it won't be any faster). That said, I find Kyoto to be a pretty huge city with lively bars, restaurants, and industry ... all the stuff of a city.
If you really want a green city - Nara comes to mind. It's not so easy to reach from NRT, but it's a very small city with lots of parkland, full of deer. It has lots of history (temples and some ruins) and it's a reasonable train ride from Kyoto, Osaka, Koya-san etc.
Osaka and Nagoya are smaller than Tokyo, for sure, but I think they still have the "big city" feeling. I find that Nagoya itself has little that interests me, but it's good for overnight trips to Takayama, the Kii peninsula, etc.
If you're looking for a "base" in the Tokyo area, I would suggest, maybe Yokohama (for a modern city) or Kamakura (for a small town with lots of greenery and historical sites). They're both in easy striking distance of NRT.
Some people have used Narita City as a base to explore Tokyo. I guess it works for them. Doesn't really appeal to me.
Other options to consider ...
Kanazawa (fly from NRT to Komatsu or take the Shinkansen)
Hiroshima (flight to HIJ from NRT or take the Shinkansen)
Fukuoka (flight from NRT to FUK)
I guess it really depends what size and style your "Goldilocks" city is - What is it that you dislike in the big cities that you are familiar with?