My experience is that when travelling with a small child you start out allocated in an adjacent seat, not across the aisle - it would make sense as it's easier for everyone that way. I get the impression it's pencilled in at time of booking.
In the worst case when you're allocated away from the child, no-one in a middle seat pair in their right mind would want to be next to an unrelated 3 year old when there's a better (aisle) seat to move into, so even if common courtesy didn't get you moved next to the child, self-preservation would. FWIW if I was on the other side of that situation and was asked if I could move, it wouldn't be a problem, I doubt it would to anyone to be honest.
I have on occasion deliberately used status to reseat myself well away from the wife and toddler on long haul, though to be fair this is mostly when we're in economy - that might sound selfish, but it's great to have a refuge seat for both parents to use. If you switch late you can often contrive an empty seat in a row too for extra comfort.