Take the train. The early morning (ca. 6:30am) train from Hartford doesn't even require a change in New Haven.
If you must drive, I would take 84 to 684 (becomes the Hutchinson) to the Cross County to the West Side Highway and find a garage somewhere to the west of the Empire State Building. IF the Merritt is clear (big IF), then it will save about 15 minutes to route that way (91 to the Merritt to the Hutch to the Cross County to the WSH).
There are also car services that will do this for about $250.