The agents refusing you boarding without visa are correct, the regulation is that a visa is required as you are not in transit . If you find that unacceptable, do a transit or do mileage runs to a different country But them are the rules, and as you have found sufficiently confusing that not all agents enforce it. As to why the rule is so, probably because the gov't employees drafting it did not consider the case of mileage runs and provide for it in the regulations.

You can get an ETA for free, but if you aren't eligible for an ETA, then doing mileage runs to Oz isn't a good idea for lots of reasons.