Given the TBIT lounge renovations are not yet complete (with the 2nd stage of Business and First yet to open) there do appear to be some additional restrictions in place -- it is often packed in the evening peaks and have seen reports of Emeralds turned away, and certainly no additional guests allowed.

That said if you were there 4+hrs before the peak and not likely to stay that long, it does seem to be a poor decision.