You don't need to stay overnight in between, but there are plenty of places along I-95 that are worth anywhere from a half hour to several hours.

Some of those are just cute historic towns, e.g. New Castle, Delaware.

Wilmington, DE is not very exciting for the most part but Winterthur is a great museum.

In Maryland, you could stop at Perryville, Havre de Grace, or Aberdeen.

In addition to the Walters, Baltimore has the American Visionary Art Museum, some interesting architecture, and good bookstores.

The other thing you might consider is an out of the way route. You could swing down through the eastern shore, for example, and go to Salisbury or St. Michael's in Maryland. Or go west in Pennsylvania instead of due south, stop at Valley Forge or Gettysburg. An interesting day trip might be a drive from Philadelphia to Gettysburg, then down route 15 through Frederick, MD and then east from Leesburg, VA to DC. (Or stay overnight in Frederick, eat an extravagant meal at Volt, then drive straight to DC via 70/270 to the Beltway, which would be faster but less scenic.)