Smallest airport I've been through in the US would be BTM, which is probably a little bigger than some others mentioned already.
Internationally, I've taken charter flights out of game park airstrips in Botswana that were pretty tiny (the type of place where you do a flyover first to make sure there are no animals on the runway), but I guess they don't count. I did have the pleasantly surreal experience of making an international arrival in a 5-seater Cessna. My wife and I were the only passengers, which didn't seem to faze the Botswanan immigration official (I'm sure it's a common occurrence, given the amount of small plane charter traffic in the region).
The smallest "airport" that I've flown into on a regularly scheduled flight would be in the Solomon Islands, either
FRE (Fera Island, Santa Isabel Province) or
EGM (Seghe, Western Province). I think the runway is a bit shorter at EGM (it was originally built as an emergency landing strip for fighter planes in WW2), but it at least has a terminal building, unlike FRE.