Really? Nothing worse? My niece and I purposely booked aisle and window on a CX flight JFK-HKG. Neither of us wanted the middle seat. Upon boarding, there was a person in the middle seat who asked us if either of us wanted to switch, which neither of us did. We lived with occasionally talking with each other, but mostly we did our own things (slept, read, listened to music, watched movies, etc). So there was little need to interact during the flight.
When I travel alone, if I don't book an exit row, I always first look for a row of 3 where the aisle seat is taken so I take the window seat (hoping the middle seat stays free). No risk in that at all.
My hubby and I do this on just about every trip. I love the window and he loves the aisle. The middle person usually asks if we want to sit together and the usual answer is "No we spend every day together, we're fine thanks".
Rarely the seat remains empty, but as we both get the seat position we like we are happy.
We don't chat too much, he likes to sleep I like to read and listen a movies, I would not think it too polite to be chatting over middle person too much.