If I was travelling alone I probably would go into the city as trains are frequent and all the action is right outside the Hauptbahnhof. However, syou are with your family and they may not feel like moving anywhere. Coupled with the fact you should allow slightly longer for a US bound flight I'm not convinced going to town is really worth it. A four hour wait can be boring I know, but rather that than the rush to get back in time.