The keys aren't worthwhile trying to do. For one thing, the traffic may tie you up for too long, for another, the more scenic part of the keys are further out and you wouldn't make it down that far and back on a good traffic day.
All a personal choice on what you would want to do, but I would hit the beach if I wanted a day in the sun (the sun may help put you to sleep in the red eye to London?), and I would hit CMX Brickell for a matinee if I wanted to relax in a dark theatre (and would eat some decent food there, and knowing me, would likely get drunk).

Rent a car and hop down to Islamorada in the Keys!

Possibly cheaper alternative to a hotel, you can usually get a fairly cheap Airbnb in Brickell. Plenty of new condo buildings with a pool and gym plus you have the shower in the unit itself. Metro station is by the rental car facility if you want to keep it even cheaper.

Laundry too if you want to save yourself a chore when you get home

There is also a bus (#150) that goes to the beach with no interim stops. Directions are in the ground transportation page on the airport

It is going to be hot, very hot and humid. Dress appropriately.

The Perez art museum is downtown, as is the new science museum. Drop by Bayside for a drink or food. They also have a ton of tour boats you could ride.
If you do make it to South Beach, Mac's Club Deuce and the Clevelander are good places for drinks. Club Deuce is a dive bar, but famously so.

If you want the keys but not the drive, go to Key Biscayne and Crandon Park for some great views.

Do you like beer? If so, hit some of the craft breweries. There are several in the Wynwood neighborhood, which isn't too far from the airport.

Would anyone recommend taking the public transportation (trains) around Miami? It looks like there is a big loop that leaves from the airport, and a smaller loop that you can connect to more downtown/Brickell area.

We have a one-day stay and are renting a car, thinking of going to Vizcaya and out to Key Biscayne. Lunch in Little Havana sounds good, too.

And we are also staying at that Airport Hilton - it's called "Blue Lagoon" which I guess gussies up expectations a bit but it does seem like a more resort-like atmosphere, and for a lot less money than most area hotels.

It's funny you mention Jupiter it can take 3 hours to get there with normal rush hour traffic.

Allow enough time getting to and from the airport, as traffic can be crazy. I've cut it close a few times and it's not fun.

I recommend El Palacio de Los Jugos, a cuban style deli with great freshly prepared juices. Locations are all around Miami.

Oh and don't forget to brush up on your Spanish, as service in english may not get you anywhere in some places.

I wouldn't risk trying to get to the Keys from MIA and back in a fixed day schedule. Best case, no traffic you are looking at 2 hours each way just to get to Islamorada not accounting for time to get the rental car and return it, and to actually do anything. Closer to 4 hours each way to get to Key West. In summer on the weekend there is no such thing as no traffic, there are times it can take an hour to go one mile in weekend traffic.

Me personally with a connection that important, I might not even try South Beach or Key Biscayne, as while they are theoretically easily reachable, when the end of the day beach traffic heads off island, traffic can be brutal.

I would try and stick with Brickell (shopping, restaurants), Downtown (Science Museum with great aquarium), and/or Wynwood (trendy urban street art market kind of place if that is your thing). Coconut Grove isn't really all that great in my opinion but you have that too.

IF you are renting a car you also have Viscaya Museum and Gardens and Fairchild Botanical Gardens in pretty easy driving distance if you want to get some nature and/or history in. If you want to go to a shopping mall Aventura would be my first recommendation.

I have no personal experience if they do day rates, but I would use the Miami Airport Hotel if they do, can't beat the convenience. Check in outside security, drop your bags, go have your day, come back, clean up, head to your flight.

As others have told you, forget the Keys idea. Lots of driving, you could get stuck in traffic, and it's just not worth it. What would I do? I would probably rent a car (or use Uber) to get myself to Little Havana for a good Cuban sandwich and a cold beer (or Las Olas in Miami Beach). Then I would head to Key Biscayne and hang out by the public beach for a few hours and read a book, or I might get a day pool use card for the Setai. Then a few drinks in South Beach on a terrace, before heading back to the airport.

I wouldn't really recommend paying for a hotel room since it is a daytime layover. Probably just take the metro train into the city. Have a nice filling lunch at a good restaurant (According to others, MIA doesn't have a really wide selection of food joints post security) . And maybe just walk through picturesque areas. I've heard that Miami is a very multicultural city. Since your flight is at 8, you would definitely want to try to be back at the airport by 5, since lines for evening international flights can be pretty long.

A couple things. Short answer - the MetroMover (smaller loop throughout downtown) is convenient to many locations including Brickell, Bayside, museums. So, if you get to the area, it is a good way moving around the downtown area. Its free and runs often.
That being said, getting from the Hilton to the MetroRail is a challenge. You'd have to take the Hilton shuttle to the airport and then the connection to the rental car/metro station.
The Metro Rail is more of a commuter rail. There aren't a lot of places it can take where a tourist wants to go. Vizcaya is about a half mile walk from the Metro Rail station. Coconut Grove is even farther. Maybe using it to get close, then Uber the rest of the way.
Key Biscayne requires a car.

Yes, we do have a car reserved, I was just concerned about crazy traffic on a Sunday (we've found in a lot of cities weekends are busier since everyone is out and about, rather than at work all day outside of rush hours). And I hadn't thought of beach traffic. That will be a bear, too. And to get the car would be the same hotel shuttle back to the airport, etc.

Maybe we'll just chill at the hotel for one day. The pool looks very nice.

Don't be too discouraged, there is plenty you can see with a car in Miami in that amount of time that has been mentioned. As long as you don't cross the bridge to Miami Beach or Biscayne there shouldn't be horrible traffic around the city on the weekend.