Agreed especially about whether the lounge has ties to the airline. Even then, if it's a long trek from lounge to gate I'd be cautious. I've only had flights suddenly leave much sooner than the delay was projected to be initially a few times. But in one instance they did a very rushed boarding when we did get the go-ahead.

I had folks from my office miss a flight after doing what the OP contemplated and ended up with a Planes, Trains and Automobiles adventure coming back home. It does suck hanging around the gate knowing you could be in more comfort in the lounge. I guess it just depends on your tolerance of the potential cost in time, cash, and/or hassle if you did miss it.