As someone that has lived in Japan for 15+ years I have never been asked for my passport. But it IS the law in Japan -- and you could be escorted back to your hotel / home to produce your passport -- but it is very unlikely.... But if you are a foreigner that abuses local norms -- I can't speak to that.....

This is like jaywalking in NYC -- technically it's illegal and you can be detained -- but in reality it never happens....

Just as an FYI - I was asked once at the airport post security to produce my passport but that was a joke - I had already left Japan lol...