If you are cabbing it.. I guess Long Wharf (Boston Harbor Cruises) is closer but its not a huge difference.
Via Public Transit, go with with Boston Harbor Cruises.
That is also only one switch vis a public transportation (green line to blue line).
Going to Seaport for Bay State Cruises via public transportation will require taking the green line, one stop on the red line, then the silver line.