It would depend on the ages and interests, where the people were from, and when they were visiting.
In general, the Freedom Trail is a must for most people. The Isabella Stewart Gardner Museum is my favorite museum in Boston, but it wouldn't interest most children. The Museum of Science, and the Museum of Fine Arts are also great, and the latter IS well liked by children. A whale watch may be a must, depending on the time of year.