14 CFR 121.585(b)(7)(i) prohibits someone from sitting in an exit row who has "A condition or responsibilities, such as caring for small children, that might prevent the person from performing one or more of the applicable functions listed in paragraph (d) of this section"
It is reasonable to assert that if a child is too young to sit in an exit row, that child may also pose a burden on the parent in the event of an emergency.
The FA is correct.