Let's be realistic. Unless you are in poor health, you can share your O2 mask with the baby. If the situation is so bad that sharing O2 would be life-threatening (extremely unlikely on short routes like CHO-ATL and ATL-BHM), then the baby probably wouldn't survive anyway.

What I don't understand is why you are trying to get a free center seat for a lap baby. Are you going to buckle the lap baby into the seat by himself?

If I were you I would move your seat assignments to the starboard side rather than play musical chairs on the plane.