I think the OP has raised a perfectly reasonable issue. Lives could depend on the speed, agility and quick thinking of passengers in the exit rows, so if these seats are occupied by people who could potentially hinder rather than help, it is valid to alert the cabin crew.