Depending on the time of year, Fuji may or may not be visible, no matter how close you are. In the winter, you can frequently see it from Tokyo. In the summer, you can sometimes pass within ten miles of it and not see it at all.
The official climbing season is July and August, but it's a real mob scene.