The Sunday family fares are great value but if you use the airport stations you have to pay the airport terminal station access fee which is around $12 per person.

As others have said, if there are 5 of you - forget the train to/from the airport. get a cab.

Car/no car is really a personal thing. If you just wanted to hit the big sites (Opera House, Harbour Bridge and maybe a ferry ride across the harbour) you could save yourself some stress and get yourself a cab to circular quay - see the opera house, walk across the harbour bridge, come back and get a ferry across to Manly. All very doable without a car (but if it is raining - this is all going to be as miserable as it gets).

I'm not trying to dissuade you from a car here (indeed with 5 of you it's probably very convenient) but driving (and parking) in Sydney isn't especially easy and if it's a nice weekend day and you are heading out to Bondi (which is as others have said, lovely) the traffic and parking can be really, really ordinary - so much so that this former Sydneysider gave up driving out to the Eastern suburbs years ago...