The OP (and anyone reading this thread in the future) should note that there is a growing trend to completely eliminate the ability to do cash payments at toll booths in the NYC area.

http://web.mta.info/bandt/ezpass/faq.html

I echo the advice in this thread to plan on using a transponder provided by a rental car agency for this specific case (out-of-town renter with no plans to ever use the service again). I travel to the San Francisco Bay Area about once or twice a year and have a transponder for that area so I can do cashless tolls when driving there.