Echoing cheltzel, the nice bits of the Panorama route are a fair drive from Nelspruit. And Nelspruit isn't exactly the most charismatic of cities, shall we say? If redeeming points is the only way to do that part of the trip then it might make sense but it will come with some excess driving. I know the area well as our winter home is 20mins north of there!
Garden route - George is a nice town, so is Plettenberg Bay. PLZ is highly missable, so is Mossel Bay. Cape Agulhas is both underwhelming and a lovely spot all at the same time.
Cape Town - lots of people love it but I'm not the biggest fan. The Cape of Good Hope is worth the drive, and don't miss the much recommended but worth it Chapman's Peak Drive. I really like Muizenberg.