You can take a bus from NYC. I have never gone on a weekday, but I would imagine it is less crowded versus the weekend. On weekends, if you want to deal with less crowds, get there just when it opens. However, even when there are a lot of people there, I've never ended up blowing lots of time standing in line - it just ends up being a zoo, and your parking spot will be remote (if you drive yourself in).

Transportation time without traffic should be around 1h15m. I usually go via Zipcar to have control over my own schedule, but there should be plenty of buses available to ferry you to/from NYC.