Sometimes this is due to incoming weather, clogged traffic patterns, changes in patterns, etc... Basically ATC determining that. It could be that you hit that window or you're stuck...could mean a long delay if you miss that tight window to get airborne. Usually there are many announcements at the gate and the agents and FA are all running around to be prepared so even if you do get there at the last second they're basically pushing back while you're still buckling in. In other words, no grace period...