It all goes back to the amount of carry on luggage people bring. For me, with just a backpack, there's almost no impact to the time it would take me to grab it on my way and go. For others that bring the kitchen sink with them, they are the ones that will slow it down.
Unless I see immediate danger to myself, I'll be taking my backpack in any evac. It contains many of the above mentioned critical items that would help me get through the next 24 hours after an event.