You can always stay at the Four Points in Norwalk. It's one of the better 4 Points in the system. There's very little to do in Norwalk. This property is right off the Merritt Parkway (cars only), which parallels I-95.
Since the Merritt parallels 95, you can avoid 40 miles of potential traffic until you have to get back to 95 headed east to Mystic.