Trains - INSANE here, I mean I'm seeing like $250+ USD for a round trip to say Kyoto or Osaka from Tokyo! ... I mean it's not that far. Yeah you can grab the rail pass for $300, I guess that's a good deal if you are going to go to more than 1 place...
As noted by others, if you were to compare the distance, speed and time, the price is quite reasonable compared to other cities and other countries. Add on to the fact that the Shinkansen is the most reliable, most punctual/time performing train system, probably in the entire world...
The fare structure isn't insane, the price is the same 1minute before departure as it is 1year before departure. I find the fares to be quite fine. Also there isn't a fare hike every 6 months as in other cities of the world...
The JR Pass is quite reasonable if one gets it.
Otherwise you can get on a 4500 yen overnight bus. Or a travel package that includes hotels and trains.
A trip to Japan is quite reasonable for a budget person minus the airfare to get to it, you can pay for a cheap hotel,hostel or guesthouse in any major city and not have to worry about paying for a low priced one and be stuck in the boonies, or being in a crime ridden neighborhood, also most places are kept up pretty well, budget places can be found right in major districts with good access to public transportation. Plus street food that's actually safe to eat!