While I consider myself a medium-to-experienced traveler, I'll admit that my risk tolerance is medium-low. If I were in your shoes, I'd probably stay up in the terminal. If my flight is cancelled, you can be that I'm hightailing it to a lounge. But if it's a delay, I won't leave the terminal. More often than not, I experience the dreaded rolling delay. But every once in a while, I've been in the lounge or gate area and received the "captain has got the all-clear to push back - get your bucks onboard ASAP!" announcement.

I guess it all depends on your risk tolerance and your backup options if on the off-chance your flight departs early without you.