It's all a false premise. There is no "unwritten rule" and it is not merely "common courtesy." Passengers are free to place their bags in any OH bin designated for their use.
There simply is not enough OH space to accommodate all of the bags on a typical domestic narrow-body and that means some unhappy people when the aircraft is full. The "penalty" falls on late boarders who are generally in higher BG's and sometimes are late connections.
The only solution is to either radically reduce the carry-on allowance for all, for some passengers, or to charge more than checked bags for the privilege. Without those, it's a simple matter of SOL.
When you get to your row and there is a placard on the OH saying "Reserved for Mr. DYKWHI's bag" then you have a beef.