Although it will likely be more expensive, I would consider doing a one-way car rental from JFK to downtown Providence -- or New London, New Haven, etc. -- dropping the car off on Friday, and then taking Amtrak and the LIRR to get back to JFK.
The OP also needs to consider how he will pay for tolls on the route: either cash if he only uses facilities that still accept cash, or the rental company's transponder, which incurs a daily fee.