I remember going to Japan the first time years ago, I could not read a lick of Japanese and got around the main cities just fine.

Do a little research ahead of time (http://www.japan-guide.com , and of course this site ) and you should be generally fine in getting around.

If you are going to the main cities, main sites, even the minor ones, again do a bit of pre-planning there's no reason to have a guide.

Sample Shinkansen (bullet train) platform sign in Japan:
http://www.flickr.com/photos/danielross/279638638/

Sample sign on a Tokyo JR train:
http://www.flickr.com/photos/manarh/121950767/