That's promising, however if I were you I would try to dig out the original email confirmation. Others might correct me if I'm wrong but I have never seen an avios hotel booking with no supplier fee. My suspicion is that the agent just quickly checked it, could be confused with the flight bookings which is only £35 the fee. Then when he/she tried to do the cancellation offline, then the computer wanted the supplier fee and never able to cancel it. It is totally my speculation and I could be wrong, still check whatever you got about this reservation.
Also it will be an uphill battle to be honest as the hotel is supplied by some 3rd party LTD, and not BA or BA holiday. So you would need a lot of time and effort. good luck!