If it's feasible, I would recommend driving one way, and flying the other (utilizing a one-way car rental MIA-EYW, or vice versa).
The drive is certainly worth doing one way; but the roundtrip drive is a bit much if you will only be in Key West for a short period of time.