I would definitely take the Ferry, but drive to Orient Point and take it from there. I-95 in Connecticut from Bridgeport to New London can still be a mess if you go that way (as opposed to take I-91 to I-84 to the MassPike which could also be congested). Just avoid most of it by taking the Orient Point ferry to New London. It's not that expensive, and worth the $, especially at rush hour.

Oh, and call them and make a reservation! Peak times it could be booked full.

https://www.longislandferry.com