It was kind of mentioned in the BA Open Doors termination letter. And it was also applied in several aspects of LHR's operations outside of T5 (which is somewhat hermetically sealed in that respect). So T1 Domestic users were not allowed the T1 International lounge, people wandering over from T2 on BA codeshares - when it was effectively one terminal airside - were also not allowed into T1 International on the basis of being in the wrong terminal. Then at T3 they wouldn't allow people to use the BA lounges if not flying out from T3 (though some people reported here they were allowed in).

In addition to TBIT (nothing to do with BA) and SFO already mentioned, I believe (but have not witnessed) that access is terminal related in ORD too. At LAX there have been reports of BAEC members not able to use either the Alaska Boadroom (wrong card) or the T4 facility (wrong terminal) when flying out of T6.

I accept that at MIA E, Premium Lounge access is perfectly allowed, provided you have status, those without status sometimes have trouble if flying domestically and are often referred to the nearby Admirals in D.

I doubt there is any T&C about this, there again lots of aspects about lounge operations are not publicly documented.