I believe Wise withdrawal fees are based on your card issuing country. For US issued cards, it's $1.50 + 2%.
Unless you actually need cash, you can just spend the JPY by swiping the card anywhere that takes cards. IIRC there are no fees associated with spending currency you already have. It's cash withdrawal that's a pain with Wise.