The agent was correct because your family is not on the same PNR as you, as they they can not take advantage of any of your benefits. That ranges from free baggage, priority boarding, preferred seating, etc. Those are the exact rules.

Now that does not mean an agent will not over look then rules and seat you together, especially as you have a child traveling with you or if there are no seat together further back. However, given that the airlines are using baggage fees as a major source of income I would plan to check all bags in your name. And make sure when using the priority line and boarding that your credentials/BP are on top.