I wouldn't exchange extra time at any lounge in ATH for some extra time downtown, therefore I'd suggest you to have dinner and drinks in Athens and then head out to the airport again.

Checking in 2 hours before departure still gives you more than enough time to check out the (rather unexciting) lounges.