IMO a doctors note to not wear a seatbelt is a virtual unfit to fly letter.
Captain was absolutely right to offload the family. If the mum was trying to persuade her kids to wear them then obviously doesn't seem (on the surface of it) too much of an issue.