I left just before 4pm and could have either made it to the Port Jefferson ferry for the 5pm ferry or the Orient Point ferry at 6pm. I opted for the Orient Point ferry. One thing that should not be minimized is that 34 miles of semi-rural one-lane road that varies in speed limit between 40-55mph (mainly 40-45). It is quite scenic which is nice and there are plenty of stands on the roadside where you could pick up pretty much any type of produce at this time of the year.
I had a nice drive to the ferry aside from one vigilante who was going 40mph no matter what the speed limit was. Fortunately I only ran into him in the last couple of miles. The drive up I-95 was actually quite smooth until I got into the greater Boston area. They're doing roadwork where I-95 and I-93 split off so that temporarily slowed me down. Plus I-93 is also undergoing roadwork south of Boston.
I think the Port Jefferson ferry may have gotten me home slightly faster but there's no way for me to gauge how bad the traffic on I-81 would have been around 6:30pm. There really wasn't much by the time I got to New London tonight. It was a nice scenic drive out to a part of Long Island that I'm not likely to visit again soon though so worth the slight detour.
But for the most expedient option one may wish to opt for the Port Jefferson ferry vs. the Orient Point ferry, depending on traffic near Bridgeport.