4 hour maybe do-able...but,
You are really taking your chances if something were to happen with the train.
This happens to us one time in NYC via ISP. One of the train in front of us got derailed off it's track. They took us around it via a school bus. It took us nearly 4 hours to get to the airport and barely made it on our flight.
It's a good thing we watched the news before we check out of the hotel. We ended up leaving hotel early to give us some extra time but still cutting it close.