Given that you're driving back to Denver on Saturday, I would simply wait until the evening to drive back (although traffic can be heavy until about 7pm). The biggest bummer is that you'd miss out on the scenic drive up the mountain on Friday night. The wild card is, naturally, the weather, so keep your eyes peeled on the forecast (starting about the Wednesday before you get here).
Have fun!