I would just pick Tokyo, a quieter part. If you stay away from the typical places like Shinjuku, Shibuya, etc you can find some very quiet and interesting areas. A short walk brings you into older neighbourhoods with nary a tourist in sight.
Cities like Nagoya, and Kyoto are still large cities.
Don't underestimate the size and scope of so called small cities in Japan. ....they are really not that small and a lot of them have the same look.
If you want quieter, go north of Tokyo---Sendai and smaller towns like Morioka, Hirosaki etc. You can get flexibile rail passes for that area..eg 5 days out of 14 etc.