Not much traffic when you arrive at 1pm

Have been on the first EWR flight (last time was in April 2011), with no delays, in CW so at the front of the immigraton queues, a taxi via the Holland Tunnel got me to Times Square between 2:30 and 3:00 pm where the most traffic was getting Downtown to Midtown while on Manhattan. So getting to Wall Street should be faster. The Midtown area around Lincoln Tunnel is more of a mess.

A 4pm meeting is achievable depending on where your hotel is subject to
- timely arrival
- limited US immigration queue

With hand luggage, I would suggest the Path train route. In my case the other half has 2 checked bags + large hand luggage for shopping.