I read an interesting piece about this story in the NATIONAL POST in Canada over the weekend which gives more details.

The Amish man in question is NOT a US Citizen, but rather a CANADIAN Citizen (born in Canada) married to a US Citizen and holding US Permanent Residence status as a result.

He entered Canada on the basis of his Canadian citizenship, but in the absence of any documentation to prove his identity the US is unwilling to let him return to the United States to resume permanent residence there.

A DHS spokesperson commented that they are even willing to permit him entry on the condition that he submit an application for a Permanent Residence Card (aka Green Card) for future use, but he refuses to do so as it requires a photograph which he is unwilling to provide.