Thanks for the responses. The reason I'm asking is because I'm going to be going back to Japan on my way to Canada later. With the routing SHA->HND//NRT->YVR, I'm going to have to enter Japan proper on my way over.
At any rate, after these responses I'm leaning towards leaving it in. I figure it's easier to turn it in and explain on my way in/through than if something shows up on their records of me about it and I have no corresponding sheet of paper to turn in.