Assuming you don't have much in the way of carry-on bags, this is definitely workable. You'll be able to cover more ground if you focus on midtown only, so consider going to the top of the Rockefeller building instead (much better than the 1 WTC observation deck and certainly better than the ESB, IMO. In any event, make sure to get timed tickets in advance.