I agree with what the others have said. The hole-in-the-wall eateries often serve very good food for a low price. Budget hotels (<US$100 for a single) are plentiful, clean, and safe. You can get by for US$50-60 in small towns. These hotels have small rooms and a bath that looks as if it was carved out of a single block of plastic, but usually a bed, a desk, a TV, and a heater/air conditioner combination. Hair dryers and toiletries are usually supplied, there is often a modest breakfast included in the price, and many such hotels have onsite coin laundries (a feature I wish European hotels would add).
Once I've paid for the airfare and the train pass, I can keep my day-to-day expenses much lower than an in the U.S.
I think a lot of the horror stories about Japan being impossibly expensive come from two sources:
1. High-level business travelers who stay in 5-star hotels, eat in Michelin-rated restaurants, and take taxis everywhere
2. Backpackers who have just come from Indonesia and can't quite fathom the notion that there are fully industrialized, modern countries in Asia.