It generally takes between 3 and 8 weeks depending on the season and the whims of the regional passport office. You can get a rush passport (about 2 weeks) for an extra fee, and if you go to one of the passport offices you can get same day service with proof of impending travel and an exorbitant fee.

I think the reason some Americans are sans passport is because until recently, it was possible to travel to Canada, Mexico and most of the Caribbean without a passport... only a birth certificate (or naturalization document) was required. The majority of international travel for Americans is to those destinations. Of course, with the new regulations requiring a passport for all international travel, there was a bump in passport applications (and processing times), but I think that has largely receded by now.