most long haul flights are boarded using contact gates, so it is more likely than not you won't be using a bus anyway. if it is A10, i would generally be there for the first bus tbh.

however, short of never flying with BA and flying with another airline like LH where all F passengers are driven to the plane, i am afraid this is just something you have to put up with.