My favorite part of Karuizawa is actually the Oku-Karuizawa area which is very scenic and bucolic with cafes, bakeries and tourist farms scattered about. But you really need a car to enjoy that area. For someone traveling by train, you'd most likely be hanging out in the old Karuizawa area, which is the main area with this one street that's lined with cafes and shops. Karuizawa is a resort area that serves as a nice getaway for Japanese people, and it's best appreciated as a place to spend 1 or 2 nights to relax and enjoy nature. It's a cool area, but it may not be worth stopping by for you.

I think going from Tokyo to Kanazawa via stop in Takayama is doable, but it'd make for an extremely long day.