Having grown-up way out East on L.I. one of my favorite places is the Soundview Restaurant in Greenport. It has the most spectacular view, jetting out over the water. Check out their Web site at:
www.soundviewrestaurant.com for directions and the menu. I have not been in ages but your question had me waxing nostalgic :-).
The food is good, traditional, seafood and pasta...the view is extraordinary.