According to Google Maps, Denver to Mt. Rushmore is about 6-hours. We were traveling from Cheyenne, and that time seems about right. Therefore, you will really only have 3-days in the area; plan accordingly. Most of the travel is going to be on good 2-lane roads, but you may have more traffic than you'd expect. Also, I don't remember a lot of rest-stops between Cheyenne and Hill City, but there are a few little towns.

(You'll drive right by Crazy Horse on your way up, if you are looking for somewhere to stop. There are also other touristy things to do along the way.)