You certainly can't use a voucher towards the fees element of an Avios booking, so that's why your second attempt failed.

But what was the reason given for not accepting it for a BA holiday booking? There has been a lot of reported flexibility for holidays, even accepting them as payment for existing bookings...