I'm the OP on this and just want to add a couple of points. As I mentioned in my original post, I frequently just keep my seat and let the crowd clear (more on this later) rather than rush off the plane. But I do believe the "right" to deplane first belongs to those in the front rows who wish to get up and deplane.
Many people say those standing in the aisles should exit first, followed by those sitting. This isn't practical. There is already a mad rush for everyone to stand up with their heads hunched over as soon as the plane lands. We don't need more of this.
Waiting for someone to get up from a window seat and retrieve their luggage does not usually delay the deboarding process. Yes, a temporary gap appears, but this is almost always closed quickly so that no actual time is lost.
If we accept that people should deboard in row order, then someone who turns sideways and puts their knees in the aisle to block line-breakers is not being "passive-aggressive," as some have suggested. Helping others to follow society's rules is a good thing, and such behavior is "passive-helpful."
I do agree that slow people should have the decency to just keep their seats and let others exit.
An Anecdote: I mentioned that I often just keep my seat and let others deboard. This was especially true when the kids were young. On one trip my wife and two then-young children were slowly getting our things together to deboard. We had allowed everyone to get off ahead of us and were taking our sweet time in getting our stuff together. Finally a stewardess came back and told us that "everyone's waiting." It seems that the plane did not deplane into a terminal but rather into some bus contraption. So when we finally boarded the "bus" there were roughly 150 or so people who did not appreciate our act of courtesy glaring at us.