This is actually a really common and unfortunate problem with the "priority" lanes. Often the priority lane only has one person working while the main lane has multiple agents. Anyone who understands basic queueing theory will see the issue, especially when you consider that priority customers often have more variability in their service time (since they are more likely to be entitled to change flights, etc).
When the one priority agent inevitably gets bogged down with a complicated situation, you would do better to be in the main line. Of course, the real solution is that if someone has been waiting in the priority lane, they should be called over by the next available agent, even if that agent was previously helping a non-priority customer. But unfortunately, this common sense policy is not implemented by AA in any airport I fly through regularly. It could just be unfriendly agents, but I suspect that it's simply not in AA's training.
I have gotten stuck in a similar situation enough times that I don't usually bother with the priority lane anymore unless there's a free agent, especially if I don't need something complicated. I just use the kiosk whenever possible and pick the fastest-moving line when that's not possible.