The Biltmore house is a stunning place, well worth a visit if you like grand, palatial houses. Commenters who don't like the house are evidently not connoisseurs of grand architecture.
With kids that young, you might want to limit your visit to the grounds. The gardens are spectacular.