Where are the most accessible places to find wild penguins in March/April?

We'll be visiting NZ for a month March - April 2017 with our 3 young kids. Our plan is to fly into Queenstown, and stay in Te Anau for a week, after which we'll begin working our way up the western coast to Kaiteriteri, before leaving the south island in the middle of April. I've heard there are penguins in Dunedin and around Christchurch, but was curious if there are any places to reliably see them in the wild along our current route?

Since we'll be traveling with small children, we'd also prefer somewhere where we can possibly watch from a distance or a blind as I understand they are easily spooked and may not come ashore if you are visible, and I would hate to interfere with their daily routine.