For me, it goes Window>Aisle>middle (is that even an option???)
I always pick the window because of reasons said before like the view, the wall to lean on and the minimal disturbance from other people. I also dont tend to need to get up during flights, especially on the quick 3-4hr flights.
Although, i am starting to understand that i may need to look for aisle seats on longer flights. As far as business or first goes, its whatever is available.