I agree with all sides of the argument here... with a twist...

the spare middle seat is not 'community/shared' space per se - it is nobobdy's. No-one has any more entitlement to half of it or all of it.

however in ordinary circumstances (with just two pax and a spare middle) - the space is, by convention, shared. It makes the experience more comfortable for both pax.

On the otherhand, while not a 'karma' or moral issue - it could well be an issue of good manners to allow the other passenger, who would be far more comfortable placing the lap child in the middle seat, the use it. (That concept of being kind/considerate to others.)