I would also agree that there is no inherently unsafe part of the Garden route, it is perhaps one of the quaintest journeys in the world. Also driving by highway from Cape Town to JoBerg is no problem and not a threat. Once you get to JoBerg you need to know where you are going as to not go to any bad areas (much like LA as stated above). As for accomodation on the garden route, there are hundreds if not thousands of B&B's in and around almost every town and wine farm on the route, we had no reservations and had no problems, most of them were quite nice.