Thanks for all of your opinions.
The one time that a taxi driver refused to take me was from the Sheraton NY Hotel and Towers (53rd st) to Westin Times Square (42nd st). He told me to just walk. Granted it was only 10 blocks and I would probably have walked normally, but I had luggages with me. I was mad

.
The last time I tried catching a cab at 3 in the morning in Chelsea, I had a hard time getting one. Every cab was occupied and people literally had to fight over each other to get a cab. I had to walk a few blocks away to avoid the "competition"
So, maybe subway isn't a bad idea after all....