Lots of dodgy streets around there, as mentioned. S.F. has a pretty bad homeless problem, IMO.
Of course, the odds that anyone will actually bother you, even on a dodgy street, are very small. The risk is there, however, so it all depends on your own risk tolerance.
FWIW, and IMHO, homeless in S.F. appear to be, on average, significantly more aggressive than homeless I've seen in other big cities. There are quite a few genuinely crazy ones. One coworker of mine was assaulted (and died a week later), and I have a friend whose mother was assaulted and suffered permanent physical damage, both by crazy homeless in S.F. in broad daylight.
I think it's important to know and stick to the safer routes. Even one particular street might be fine on one block and dodgy on the next.