It's the crocodile juice that makes it taste so good!
But seriously, NYC water comes primarily from several enormous reservoirs in the Catskills, where water is fresh and clean, and it's brought to the city through a massive tunnel system that keeps the water moving so it doesn't pick up impurities or sit in pipes for very long.
Read all about it at
http://www.nyc.gov/html/dep/html/watersup.html
I just moved to a new apartment in an old building, and had my tap water professionally tested just to make sure the pipes were good. It was perfect in every category.