I visited Hiroshima on a day trip from Kyoto. It's definitely do-able; in fact, we had time to visit the Peace Park and then catch the ferry across to Miyajima in the afternoon before catching a late train back to Kyoto.

It's only about an hour and a half from Kyoto to Hiroshima on the shinkansen: leave Kyoto early and get there in time for a late breakfast and then get a late train back after dinner.