The SFO leaves together with a bunch of other flights. Regardless I have not seen lines above 10min at passport control for the past many months. Even at peak hours. Say you want to be at the gate 40min prior to departure. It is a 5min walk from lounge to passport - and only a few min after that. Put a 15min buffer in. So leave the lounge 60min prior to departure and you should be OK.