Applying for a NZ visa with a new passport may help. I can't see that hurting, but I could foresee circumstances where it could help.

With regard to applying for a UK visa, a new valid passport is highly unlikely to make a difference since the checks run for visa and other permit applicants are run based on the biodata information. That biodata information is presumably rather consistent across passports unless a name change took place; and even a change of name doesn't guarantee an old overstay won't be noticed.

What I am curious is about where and when the OP dealt with exit control on leaving the UK. The UK isn't generally known for having permanent staffed exit immigration control stations in the UK airports of exit.