So, I plan on moving back to the US from Japan with puppy. According to the US Customs website, I need to provide documentation of a rabies vaccine in English or with an accompanying English translation.

That just leads me to wonder what is acceptable as English documentation. :confused:

If my Japanese veterinarian can write and sign an explanation of the document in English, would that be enough? Do I have have to take the vaccination certificate to a translator approved by the local consulate or embassy? Could I just translate it there myself at Customs and sign some sort of affidavit?

Why don't you email US Customs and tell them your exact circumstances and ask what they will accept in your case? Keep this email documentation for when you travel as backup. @:-)