The issue is not simply where you stay, but where you need to be. To be very clear, the area may be in total gridlock for hours at a time. Thus, it is not about planning 30 minutes for a 5 minute drive, but rather about whether you need to make alternate plans because an entire area will be closed off for hours or the traffic is so bad that you will simply sit.
Staying elsewhere means that you need to get to Annapolis and that presumes parking. A better approach if you truly must go at this time is to find a hotel within walking distance of your destination and then stay put.