Severe congestion bordering on gridlock around the central terminal area was a reasonably regular occurrence when the tunnel work was ongoing a year or so ago and there were temporary traffic lights around the entrance/exit of the Central Bus Station - I certainly had one occasion where it was clear nothing whatsoever was getting in or out, and in the end it was quicker for me to take the tube to T5, then get on a bus to go directly back to my Bath Road hotel to avoid the tunnel (so a diversion of about 4x the original distance).
I've not seen it recently, however - so it sounds like you were probably just a little unlucky.