I don't know how this would work timing wise, but you can do this in one day. You could take the GSP in NJ down to Cape May, take the ferry to Lewes, DE which would be a nice minibreak, then drive down the DelMarVa to the Chesapeake Bay Bridge Tunnel into Virginia Beach and then continue down to the Outer Banks.