Random thoughts:
After the White Mountains, why go to New Haven? I don't see the attraction of New Haven.
Have lunch in Portsmouth but I don't see why you don't continue to Portland. Maybe add a night in Portland and do a day in Coastal Portland area (Portland itself, Cape Elizabeth for the Portland Head Light, Peaks Island, Freeport at night) and a day inland toward Sebago Lake. Or take that extra day/night and see Camden including Mt Battie.