I suspect you will get some very good answers shortly, but let me add my two cents here now.

I think for western music that is exclusive to Japan, such as Jazz artists that do CDs here that are not generally released in the U.S., the big stores like Tower will be a good bet. They have access to everything and can afford to stock it all. I have had good luck with some Benny Green (the pianist) CDs like that.

For Japnese stuff, there will be others who can better advise, but I suspect that Akihabara may be a destination you will want to mark on your train map.

Digital media is a lot of fun here. You will dig it.

Mike