Maybe you should change friends...
BA plays it by the book, so there is no way they would turn down a CX DM at the F lounge if he/she is flying BA, even in Y. But they do require all pax (BAEC, MPC or others) to show their cards. If you do not have it and are a BAEC Gold, the lounge dragon can check it in their computer system, but not sure that they can do it for CX DMs.
Galleries lounges (J & F) at T5 are wide open to any OW saphire or emerald. At T3, the situation is a bit more jammed, and there is a J and F CX lounge. So if you are flying CX, BA might be using the excuse of capacity to turn down a CX DM at its BA galleries and send him to the CX lounges. But if you fly BA, then they cannot use that excuse.