I would spend more time in CPT. Just be wary of weather. We were just there (albeit winter) and the cable car for table mountain was closed all 4 days of our trip and our robben island tour was cancelled due to wind. We didn't have so much flexibility in our plans (we had a great guide for part of the time) so we weren't able to get to robben island at all.