Thanks for the feedback.
It won't be the end of the world for us if we run into bunches kids or other crowds for that matter. Part of the experience is the crowds I guess (as it is for most urban place we visit).
We're not set on going around Labour Day. We just want to go when it's still somewhat warmer and the days are also still decent daylight. Going when we can visit/hike Mt. Fuji would be nice too but I'll save the logistics of that for another thread (yes, I know Sept is out of the regular climbing season).