Sorry, forgot to comment on Colona Castle. Never heard of it (which doesn't mean anything). But it seems rather stranded in nowhere. And if my choices were Muizenberg or Hout Bay as my nearest "village" I would take Hout Bay every time.
Also, Muizenberg (and judging from its position, Colona Castle) is right in the path of the South Easters that often howl around the Cape in early and mid summer.