Yes, BA is well within its rights to use the return ticket to cover the cost of repatriation. I don't think it even matters what the cause of the refused entry is. If a passenger is refused entry, then BA can hold the passenger liable for all costs or fines associated with the refusal. BA's conditions of carriage state they can reclaim these costs "from the value of any unused part of your ticket, or any of your money we have in our possession".
I strongly suspect that the OP may no longer have a valid ticket for the return leg. If the OP's ticket remains valid, then I would expect BA will be seeking the cost of the repatriation flight.
Could be interesting to see what is meant by "costs" here. Is it the notional "price" of the one-way fare? Or the actual amounts BA had to spend / disburse - e.g. airport taxes, and the marginal cost of one more passenger on the flight?