Actually this doesn't annoy me at all and is the way I expect things to work. I generally expect the people seated in rows in front of mine to deplane before I do and I expect to get off before those seated behind me. So I wait for those in front of me to pull down their bag and then I take my turn to get off. There are exceptions to this but absent a good reason I expect to deplane this way.

I find it annoying when I want to get my bag down and people from behind my row push forward in the aisle trying to get off quickly, thereby blocking people's ability to get to their bags.

I guess it's all a matter of perspective.

On my last flight I started to enter the aisle to take my turn to get off when a guy from a row behind me tried to zoom past. I gave him the stink eye and he backed off.