Toughest bit of Fuji-san climb is waiting on old folks ahead

If I recall, you lived in Japan, so perhaps you've already climbed from one of the 5th stations upward. If so, maybe it will be more strenuous below.

I did an overnight hike from one of the fifth stations, and I thought it was pretty easy (I don't even recall any scrambling), and that the only difficult part was the annoyance of waiting on groups of older Japanese (mostly women) in a couple very narrow spots. In fact it was so crowded that I probably could have done it without a headlamp and relied solely on light from others. I would advise hiking sticks for the descent. I sank and slide quit a bit on each step down for the first couple hundred yards.