What documentation is it that you think tells one or the other country of the second nationality? It is not recorded on a Koseki Tohon nor a Birth Certificate.
In Japan, you have to disclose any foreign nationality you have every time you apply for a passport (there is a space on the application form for this. If you fail to disclose, you have committed a crime. That said, as far as I know, they can't do much if you disclose a foreign nationality as long as you are in compliance with the Japanese Nationality Act.