I may be a little late to this thread, but I would do:
Day 1: Arrive in ZQN, immediately drive to Te Anau; overnight in Te Anau
Day 2: Drive to Milford Sound and check out different spots along the way (Lake Marian and Key Summit come to mind, but they take a few hours to complete); overnight in Te Anau or Milford Sound
Day 3: Drive back to Queenstown and explore the town; overnight in Queenstown
Day 4-6: Overnight in Queenstown for the rest of your trip; take a daytrip to drive to Wanaka (early) and do Roys Peak; the remaining time, explore Queenstown, taking one day to do Ben Lomond