I also recently visited the TWA hotel, and while it is accessible from baggage claim at T5, the walk from the Airtrain to that point is quite lengthy. The only way I would consider it in your circumstances was if you got dropped off directly at the hotel entrance to start. However with your concerns and having access to the Flagship Lounge, I would also likely concur you should just proceed right there. You mention visiting NY often, so just plan to stay the night at the hotel on a future trip. It is definitely worth seeing when the time allows.