I had a chuckle at the comment about not being able to recognize the hosts among the people wandering around the lobby area. NYC is indeed the land of PIBs (People in Black)

Every time I went in and out, there was someone at the door. But, I could see how there could be an issue with taxi access, since 1)the hotel isn't "landmarked" with signs and 2)the entrance is off to the side of 5th Ave. Maybe I was incredibly lucky my one time I asked for a taxi - it was hailed immediately. I didn't wait at all.

Different hotel setups appeal to different people. Some will love the Andaz, some won't. The only way you will know for sure if you will like this place is to try it. If you like it, you have a new place to stay. If you don't like it, you know that Andaz isn't your cup of tea.