It’s very clearly laid out in the CoC:
The suitability of carry-on baggage will be determined by American.
...
At times, additional limits may be placed on carry-on baggage based on the main cabin stowage capacity of specific aircraft, which can result in carry-on bags being checked in the boarding area.
Yes, it’s arbitrary. Yes, they’re allowed to do this.