We do this, have about a 50% success rate. Have never encountered someone unwilling to switch to either (their option) the window or aisle. In the unlikely case that we do run into that, I am sure we will all survive. (We only really do this on short haul flights).
If in Y on a long haul, I always try to choose an aircraft with 2-4-2 or 2-3-2 seating and get the pair and avoid a third person altogether. (767/A330/340 now that most 777 operators are going 3-3-3 or 3-4-3).