I have to say I've always preferred the window. Ever since my first flight as a child (sooooo long ago), I've had my nose pressed against that window from taxi out to arrival many times. I love to watch the scenery go by.
When it's a long flight or overnight flight, I sleep better in a window seat as I can lean against the wall.
Also, I usually don't have to get up that often, so I don't have to bother the person in the aisle or middle.