as you say, BA would bus pax back to the 'intended' terminal even if they arrived at a stand at a different terminal.
BA's ops at LHR are much more 'segregated' like that (by destinations). I experienced T6 at LAX for the first time on an AA flight last week and not sure if that's as published / set in stone as it is with BA.