In my experience the Visa exchange rate tends to be 0.5%-1% worse than the average rate for the day taken from various sources such as xe.com and oanda. The Mastercard rate tends to be anything from 0.25% better to 0.5% worse. Furthermore, xe.com and oanda live rates differ from each other by several basis points anyway, so how do you really know what the "mid-market rate" is other than what Transferwise says it is.
I don't know where you are getting 2% from, but if that is really the case for your cards you are getting ripped off by someone.
For all its claims of transparency, Transferwise makes it very difficult to determine its fees in advance. For a USD to GBP transfer, I know that the fee used to be around 0.5%, although looking at their website now, they claim they have lowered their fees to 0.37% plus about $1 USD.
2% was based on what my bank charges me on SWIFT transfers, not what I pay on ATM withdrawals. In fact my bank charges around 1.5% for SWIFT, but it varies. and it's mainly ZAR to INR, so between two 2nd tier currencies. Transferwise tells you explicitly what the rate will be before you confirm the transfer.