If you don't take a tour and are looking for something to do we went to Jenolan Caves and really loved it.

There is a great hotel next to it called Caves House. It's looks very old, all wood etc, but it is beautiful. We went in winter and it was cold, but they had the fire going inside. Really great place and I loved walking through the caves.