That's a lot of doubling back. You're losing a lot of daylight with fairly inefficient planning. Maybe try a "V-shaped" route as suggest by Mwenenzi of HKK-ZQN-CHC.

In Queenstown, do the Ben Lomond Track. It track starts right in town; skip the first leg and take the gondola.