I'd say certainly Guyana as a whole would be safe; Georgetown itself is a bit rough in some areas, but keeping to the center of town, and using some common sense should be Ok. There were three of us traveling together, which may have helped. We did take taxis at night, mainly cause there weren't any streetlights (very dark) or raining, but prices were reasonable. There wasn't any tourist hassle.

Kaieteur is certainly worth a visit, and the south part of the country (near Brazil) is also supposed to be interesting with cowboy culture. We didn't make it down there though.