You have enoug time to check out downtown (the main plazas, san francisco monastery, cerro san cristobal, eat lunch, or whatever you want), go to a museum if you;re into that, then head to miraflores in the afternoon and hang out in the parks along the cliffs or walk to the beach, sunset happy hour at larcomar).

Miraflores is fine at night, the area around the airport isn't. It easy to get a taxi, you can order one from uber. I think its 55 soles from miraflores. You don't need more than 45 minutes at that time of night (should be a bit less)...its an easy ride up the circuito de playas.