Times Sq is to be avoided as one must arrive in the afternoon to be penned with no bathrooms as you said above.
A wide area gets penned off between 5th or 6th Ave to the east, uptown to probably 57th St, and downtown to maybe 34th?
Pick a hotel away from that area. You can go out to dinner, see an off Broadway play, go to the Philharmonic or the Metropolitan Opera at Lincoln Center, whatever you want.