I don't think it's too packed. And I disagree that Takayama and Kanazawa are too similar to bother visiting both. Both cities are charming, but in quite different ways. If you're going to cut anything, I'd reduce your time in Nagano prefecture. I think you could spend a full day in Matsumoto and then be ready to move on. The castle is worth going out of your way to visit, but the other tourist sites are not in the same league - and not in the same league as what you could do with an extra day in Kanazawa, for example.
If you were thinking of Niigata, I'd say that you might do better to skip Niigata and possibly make an excursion up the Noto peninsula from Kanazawa. Very rugged coastline and beautiful scenery up there.