Section 75 refunds: credit card protection’ - MSE
Since Amex have already refused you may have to go to court to get your money back.
You will likely win or get it settled if you go down that route.
BA and others have a history recently of just stonewalling a lot of things until you go down this route.
Most people won't be bothered so I'm sure it saves them a lot of money.

You need to say you do not accept the Avios and get the to send you a reply to say that is their final ruling so you can progress to other means.
You just want your $500 back right? Tell them that, keep it as simple as one or two lines, don't make a long thing justifying stuff. Make it as simple and foolproof as you can.

I paid $500 for excess luggage on xxx despite my status indicating that luggage should have been free.
Please refund my $500.
This was a BA holiday purchased on a credit card so BA is responsible for the required service not being provided.