I would wonder why you would travel on vacation with people that you didn't spend any time with...
I have traveled in groups, but as others have said, we were together or at least partially together. Never were we 6 people going 6 different ways.
Sorry it was an issue for you, but I can't see it as an issue with too many people besides you.