Memorial Day weekend is the first weekend of the summer and, traditionally, all the Chicagoans head to their cabins in Wisconsin on Friday night. On Monday, they will all come home. Traffic from Wisconsin back to Chicago starts backing up at the Wisconsin Dells, which is way north of Madison and crawls all the way back. It's no better if you try to cut across Wisconsin and come down from Milwaukee.
I'm actually not joking when I tell you to plan for around a 5-6 hour commute for what should be less than a 3 hour drive. And that may be cutting it close.