I have been looking into this separately. Toyota Rent-a-Car indicates that International Driving Permits (issued by Geneva Convention signatories) are acceptable with passport. Japanese translations are needed for certain other cases (including for Swiss Drivers Licence holders, which might explain your case).

Toyota’s requirements are here but I would be grateful for further first-hand stories.

Back on topic, I had initially ruled out this property for an upcoming stay as it seemed too remote — booked the HR Hakone instead but the recent posts have me reconsidering 🧐