It would be cheaper to rent a car pretty much anyplace other than New York and the airports. Take the train to Providence as suggested, you avoid any traffic that way.

If you want to drive up to hit the casinos along the way, I would take Metro North trains to Stamford and rent a car there, both Hertz and Avis are at the train station, or in New Haven Avis is at the Station and Hertz is very close I believe. You could take Amtrak to either of those too, while it's a more comfortable ride, it costs more to do so.