I am always in an aisle seat and have never had anyone "rush" past the my row to get in front. I wait until everyone in front of my row has gotten into the aisle before I step out and let the rest of my row out in front of me.
I have noticed some stay in their seats making it clear they will leave when the plane is empty so, of course, we all move past them.
I would feel it rude if people just flew out of their seat as soon as the seatbelt light went off to push ahead of everyone else. I am usually trying to catch a connection, so would LOVE to just push in front of everyone...but it is bad etiquette.
This is just my opinion. I am not a frequent flyer, so I will be watching to see if I have this all messed up...lol