I have also never been asked by the police in Japan for my passport or any other form of ID. I've heard stories, but these seem to mainly involve checks of bike registrations, rather than just random checks of pedestrians.

I also find it highly unlikely that you would be "surrounded by a dozen undercover agents" for no reason other than that you didn't have your passport with you...