a ladyfriend and I did Kinugawa Onsen (up in Tochigi) and surrounding areas in late October. While it was chilly, it wasn't uncomfortable. Bring a sweater or jacket and you should be fine. IIRC we did a boat trip down the river and a day hike up it. As far as it being too cold, I kinda liked the hot water and cool air :-)

_m