I don't pay to pre-assign whatever seat I like on any of the carriers I fly on regularly, but you're right, it tends to be unavoidable.
I don't find babies to be a big problem, but screaming toddlers can be quite difficult.

However, if you avoid being seated in the centre 4 (of 2-4-2), then I'd say you tend to have a better chance of not being seated right next to a baby.

Not that it always works. The entire J cabin on my last night flight was woken up by a screaming toddler. It was quite an interesting sight as everyone ended up being woken up, including me who was seated right at the other end of the cabin.

Sometimes, one just has to live with the inevitable, but trying to minimise the risk is still desirable if you want to improve your chance of getting a better sleep.

Another thing of course with aircraft without direct aisle access is one's own mobility. With good mobility CW window seat probably won't present much of a problem at all unless someone is very short, but I can see that it can be a problem otherwise when trying to access the aisle. On the other hand, if you are a light sleeper, a window seat is likely to be better (I can sometimes feel the floor flex in an aisle seat as some people walk).