Until the early 90s, I would generally stay in hotels in Hillbrow or (less often) downtown, and walk around and between the two alone at any time of day or night. They only precautions I took were to carry nothing whatsoever of value on me, and to dress in a T-shirt and shorts which made it obvious to even casual observers that my pockets were quite empty.
That was before the mad flight to the northern suburbs, downtown was still the place to stay back then. A few years later, things went downhill really fast in a very short time (hence that flight to the suburbs). Hotel guests in tears in the lobby, having been mugged in broad daylight, folks brandishing firearms etc. etc. Most of the hotels soon closed for lack of guests, and downtown became a no go area. Things have slowly improved since, and I would not hesistate to walk around there again, at least in daylight, but I cannot think of any reason to do so. I have no business in JNB, just use the airport as a transfer point.
Johan