It seems a bit pedantic for the lounge dragon to deny entry in this case as only the lounge obsessed or those with time on their hands are going to go to a completely different terminal just to sit in a waiting room.
Having said that, if the BA facility was only really designed for their twice daily services, it would be asking quite a lot for it to be the defacto lounge for all American services. I know their Sapphire / Emerald cardholders cant use Admirals clubs when on a domestic itinerary - but can they use other One World lounges?