The Hampton Jitney is fairly popular
www.hamptonjitney.com
Motorcoach service from NYC to the Hamptons.
The Jitney drops you off on Main Street in East Hampton (if I recall) and you can walk around, see the beach, and eat without having to cab it anywhere.
If you decide to tour more, you are going to have to cab it, and then get back to the Jitney pick-up.
There are other ways to get there, including plane/helicopter. However, your cost will go up very sharply for this mode of transportation, and you will have to get transportation from airport or heliport to wherever you are going.
As for "Is it worth the day trip?" I can't answer that. We are two different people.
It may be worth it to you. I never recommend a "go or no-go" without knowing what you want to do or what you expect to see.
Here is a "Best Weekends" link you may get some info from:
http://www.bestweekends.com/thingsto...THE%20HAMPTONS
Good luck.