I live in suburban Washington DC on the Virginia side. My family has a beach house in Sea Isle City, NJ. We go up there a couple of times each summer - leaving the DC area around 730pm. Leaving after rush hour makes a huge difference as the drive takes us 3h 40m instead of 6h with traffic.

For the return, we always leave the shore area by 2pm as the traffic exiting the beach areas really builds up after 3pm.

It's always faster to head up I-95 and cross the Delaware Memorial Bridge vs. the Cape May-Lewes Ferry. Take the ferry and you must leave DC earlier or you risk missing the last ferry of the day. With traffic, slower route, waiting for the ferry and crossing, it becomes a 7h journey.