The Change Fee is Waived within 24 hours cooling off. You only pay a difference in Fare and Taxes, ditto Avios, as in this case. You shouldn’t have been told you couldn’t change it and have that Fee waived. The only issue you may encounter is if you are changing to a difference Avios Zone which means you do have to cancel.