You'd probably be fine in either direction around noon. The other time frame is creeping into rush hour. I wish I had more experience with the drive. Unfortunately, most of the time I've driven back to the FLL area was closer to midnight.

You might consider calling/emailing a shuttle/limo company. They would have a much better idea on traffic between the airports. I use Go Airport Shuttle out of FLL. They are the "official" shuttle service at FLL. You might be able to find someone cheaper...but, I like the fact that there is a person outside of each terminal when I arrive.

http://www.go-airportshuttle.com/

Even if you don't book with them, they should be able to provide a better idea when it comes to traffic and transit times.