I concur with NM, even thought the QF lounge is physically close to the normal Qantas gate, passengers still have to head back to the terminal's central area and travel down the lift/escalators before turning 180° and walking back past the lounges which are now 1 level above.

With a 4yo; I would head to the gate 45 minutes before scheduled departure. (Without a 4YO; I would still head 40 minutes beforehand)