The Four Seasons which used to be the Intercontinental is fine. The Westin, no. 1 Martin Place, (not in the top class) is centrally located in the old post office building and has some nice and modern rooms.
But, hey, the Observatory hotel has more charm and is IMO even better. Small Orient-Express hotel, great service,...
Try it