The Harbour Bridge Pylon is a much cheaper alternative to the Bridge Climb, but still has great views and it has an excellent series of displays detailing the engineering and building of the bridge.
Pubs near there that I like are
The Glenmore and
The Australian but nearest would be
The Mercantile which has become something of a generic Irish pub.