I think most little boys would love a visit to the "Egouts de Paris" (the sewer!).
When they're a little older, a trip to the catecombs could be in order - it might be too much walking/stair climbing now. The Cite des Sciences is fun too.

This information comes from a summer spent entertaining an 8 year old in Paris about ten years ago - so it may be a little out of date.