To cool off, do an overnight at Mt. Koya (Koya-san). The high elevation makes it a great spot for escaping the heat (which also means that you should make your reservations early).

There is a direct train from Osaka's Nankai Namba Station, probably the easiest way to get there.

If you go to Kurashiki (you'll want to take a walk along the shaded canals in the heat), be aware that Shin-Kurashiki, the station on the Shinkansen, is not near the historic district and is in fact several miles west. You have to transfer from the Shinkansen to the San'yo Main Line at Shin-Kurashiki or (to avoid backtracking) at Okayama.

I second the advice to pace yourself in the heat.When I was in Takamatsu (another possible day trip from Osaka), I walked the eight or ten blocks from my hotel to the Ritsurin Park (a famous formal garden) and survived the walk through the gardens, with the help of a snack stop, but when I finished, I realized I simply could not face another walk back to the hotel, so I grabbed a cab.

Japan's heat is not like that of New York or Boston. It's more humid, more like New Orleans, and Kansai, especially Kyoto, which is surrounded by mountains on three sides, has always seemed worse in this respect than Tokyo. I have always marveled that Japanese people in the old days wore multi-layered kimono instead of sarongs.