Tough situation, but one that I have been in while awaiting takeoff from BOS. We also watched an entire movie awaiting de-icing. It is a tough call to disembark an already loaded plane, mainly because there are many instances where the expected time gets shortened abruptly, ie. a flight in the queue gets cancelled and if you are ready to go, then you might get an early call-up. If you are not sitting ready to go, somebody else gets the slot.
Most pilots want their plane set and ready to go rather than with pax wandering around the terminal. They have already done the pax-baggage match etc.