Larger banks in the city itself should offer exchange services, I've done so at a Hokuyo Bank in Sapporo before.
I've found some posts saying Nanba currency exchange in Osaka is good as well.. current rate is 147jpy/USD, spot rate is 150jpy/USD today so it's not that bad.
Avoid anything run by Travelex tho.. I found them to be the worst rates.