Safety; the regulators wanted there to be no financial incentive to pack people onboard a 50 seater regional jet busting safe weight and balance numbers. The smaller aircraft are subject more to the whims of weather, making it more difficult to plan for the 100% case. In contrast, mainline aircraft are usually only subject to weight and balance issues as a result of - as you say - poor planning (like flying a classic A321 to Hawaii or out of SLC in the heat of the summer).

In the new world where airlines try to never IDB passengers, it's more or less a moot point anyway; now that the UA incident has incentivized the airlines to only use volunteers at any cost, the financial incentive is back to bust safety margins in favor of volunteering out fewer people.