Porto Rafti isn't that far and
Varkiza as well (both best reached by taxi), but at this time of day no way you can have lunch, you'll have to satisfy yourself with coffee or a freshly presses orange juice. Careful: getting there is easy, for the way back you should call for a cab as both places (especially Porto Rafti) are not exactly dead but commatose during that period of the year. Still, your idea is good, I've done it myself a couple of times...