I have never been a luxury traveler and prefer not to spend a lot of money on hotels, so most of my Japanese-style experiences have been in minshuku, which are like bed-and-breakfasts, only you receive both dinner and breakfast.

In some parts of Japan, such as the village of Tsumago, they are the only option. They give you a glimpse of the local culture, a chance to meet other travelers over meals, and a get-away-from-it-all atmosphere.

In the cities the OP mentioned, ryokan are just an option. (There are few minshuku in the large cities.) As others have said, it is a matter of whether you want to sleep on the floor, eat purely Japanese cuisine, take a Japanese bath instead of a shower, wake up according to the ryokan's schedule, and vacate your room at certain hours of the day.