If you're just spending a couple of days in the Alps (say, one day in Kamikochi, another day doing Kurobe Alpen Route), then Matsumoto is a good base. Takayama is another potential place to consider, especially if you're interested in Kamikochi and Oku Hida area. Matsumoto, in particular, has a lot of reasonably-priced accommodation options. But both cities get hot and muggy in the summertime almost like Tokyo. And if you want to spend a couple consecutive days exploring Kamikochi, then stay in Kamikochi since it's a long bus ride from either Matsumoto or Takayama. If you want to spend days trekking up in the mountains, then camp and/or stay at mountain lodges.

Another potential place is Hirayu, which is much closer to Kamikochi and may, in fact, be a cheaper alternative to staying at Kamikochi. Hirayu itself is sort of a little town built around a bus terminal and there's not much there. However, Hirayu's onsen quality is top-notch and the area is noticeably cooler than Tokyo, Matsumoto or Takayama.