Well, you can always just ignore those folks' opinions.

I think that I see the opposite reaction almost as frequently - e.g., "Oh that is waaay too much time spend at ________. You'll be bored!" "Why are you staying __ days in _____? All the interesting stuff can easily be done in a half day"