When you renounce your citizenship, you get a Certificate of Loss of Nationality. You should carry it with you to prove that you are not a US citizen when traveling in the US, because without it, anyone whose foreign passport lists the US as place of birth is assumed to be a citizen.