No personal experience of the public transport system, I've always got a rental while i'm over there, but friends that live there say the public transport system is excellent. Taxis seemed cheap (compared to Auckland rip-off rates) the few times I've used them.

Glenelg is worth a visit on a nice day, and Hahndorf is a bit touristy but still nice, and I hear good things about the wineries in the area if that's your thing.