You've got a lot of variables in that scenario - but the short answer is yes - planes take off every day with passengers standing outside the jetway (usually upset).

In your situation, I would interpret the message to be that the delay is indeterminate and that if the captain gets clearance to push back he will immediately do so and those who have stepped off the plane may not be able to get back on. Whether that clearance will come in 3 minutes or 3 hours remained to be seen. But the captain wanted folks to know that there were no promises and that those who chose to step off took their chances.