Inbound we have a middle and window in E+, so we'll just have to deal with that.
You might want to rethink this. If I were the aisle seat PAX, I wouldn't be too happy sitting in your row. No reflection on you, it's just that you will take up more room with two PAX holding babies, plus you will very likely be getting up and down a lot for diaper changes and aisle walking. You really should consider booking opposite aisle seats. Same for your aisle and window seat booking. In the very likely chance the middle seat is sold, I'm assuming that you would move to the middle and window seat, presenting the same dilemma for the poor aisle PAX as your return flight.
Good luck. At least it's not an international flight. And bring twice as many diapers, bottles and changes of clothes on the plane as you think you'll need.