While the agent shouldn't have been rude about it I do understand a little bit. By D-45 all the everyone is hovering around the gate, and as soon as the boarding door is open people rush to get to the front of the line.
While of course you *do* know how to read a boarding pass, I'm personally surprised by how many people seemingly *don't* know how to read a boarding pass and insist on boarding out of boarding group.
I think Jetblue sums it up quite nicely in this video: