Scratching my head a little on this, as I'm thinking I've done that with no bother. I got into Arrivals lounge in May, after I flew into T3 on US in J.

I had an Avis rental to pick up at T5, so with hand luggage only, I took Connections bus to T5 and cleared passports there.

Went to Arrivals lounge and had a very easy time with lady on door - smiled and was prepared to explain my logic, but not necessary - just showed her my boarding card, and my GC. Never seemed in doubt. Not sure why we had such opposite experiences.