Weekend traffic can be busy, plus it's spring break time, so there's additional vehicles to account for. Normally, I'd say plan on at least 3.5 hours on the highway. If there is an incident along the way, chances are traffic can come to a standstill for a while. Keep that in mind if you are coming back to catch a flight Monday. It'd be better to leave earlier Monday and if there's no traffic problems, get back to Miami early and enjoy lunch somewhere.
Duval St. will be a spring break zoo, especially on Saturday and Saturday night. That means parking will be tough to find. I'd suggest heading to some of the local establishments "off the beaten path", hop on one of the Trolley or Conch Train tours, and/or taking a stroll thru the Key West Cemetery and reading the tombstones.