Have you read what the detailed fare rules state? There may be wording to the effect that if a passenger is ill and unable to fly blah blah blah and traveling companions as well.

Perhaps they do, perhaps they don't - if you have not read them, one possibility is to go to the refunds site and add your ticket number, reading the rules before you actually request a refund of course.

But it sounds like in the end you will have to throw yourself at the mercy of the airline; some agents are better with this than others, but an airline would not want a passenger who might be contagious to fly, and I can't imagine they'd want to force you to fly with a sick baby. Be sure to ask if the physician can sign a statement stating the baby is too ill to fly. And in my experience, this is one of those occasions when being EXP (and calling the EXP line 1.800.843.6200, perhaps using Skype or other VOIP, etc.) can make a difference. (Be aware there is a developing winter snow storm in the northeast USA once again...)

Good luck!