Regarding Great Ocean Road, I did this as a one day affair back in April of this year. What you are trying to do is feasible.
I picked up my rental car at 8am and was back before 5. I drove out on the GOR, and took the inland route back. It took me a little more than 5 hours to make it out to the apostles, allowing for a stop for lunch in one of the charming towns along the route. I had the nav system take me back through the inland route, which took about three hours. The return drive isn't dramatic, but it is a pleasant trip through rural areas.
You will only have time to stop at one of the towns on the way out. I don't recall the name of the one I stopped at, I just saw one that seemed lovely and stopped there. You can't lolly through each town you come across and make this a one day journey.
I spent about 30 minutes at the apostles.
Be careful of your speed on the road. The nav systems warn you where speed cameras are, but beware that Australia is sort of the opposite of Germany in terms of driving pace.