Sure, that will work.

If you have a brokerage account or 401K then see if they offer a checking account with debit card that refunds the ATM bank fees. The bank fees for the Japan bank are minimal but your home bank fee might be a bit of a spanking (USA banks, anyway)

At the ATM you may be given an option to do the transaction in yen or in your home country currency. Always choose yen. The other option is a ripoff called dynamic currency conversion.

@bocastephen mentioned using orphan cash to pay down your final hotel bill. That is an excellent idea.

Sometimes my final stay is on hotel points. In that case I just hold on to the cash for a future foreign trip. JPY are easy to convert all over the world. I carry the cash (paper money) to convert in case I have any issue with using an ATM.

One time, I won big at Woodbine and was unable to convert the CAD to USD before my flight home. Used it to buy yen in Kyoto.