Maine has perhaps the highest concentration of these names in the USA. The World Traveler Sign is famous:

http://www.roadsideamerica.com/story/10515

The various towns have different reasons for their names. One recurring theme seems to be sympathy with foreign countries: Mexico, ME, for example, was named in solidarity with the Mexican fight for independence from Spain; Denmark, ME was named in sympathy as well, as the British Royal Navy had previously attacked Maine and then moved on to attacking Copenhagen by the time of the naming. Same story for Peru, ME, too - those darn Spaniards again.

But there are other reasons, too - Norway, ME originally wanted to be Norage, but adjusted once Massachusetts rejected the original name for reasons lost to history. China, ME was named after the common Christian hymn, "Jesus Loves Me", which appeared in some 18th century hymnals as "China". And Poland, ME got its identity from an Indian chief named Poland