I've been that person in the middle and the people on the window/aisle didn't even offer a switch, but assumed they could act as though I didn't exist (e.g. talking over me as though I wasn't there etc, making sexist comments about the female pilot over me, a woman). Fortunately, I had a newspaper on me which I would normally have read folded up, but since it was a broadsheet, by reading it not folded it provided a suitable barrier to their chat, and I added my own views on the sexist comments they were passing. As a result we sat in stoney silence for the duration of the flight. Oh, I also determinatedly refused to cede the arm rests (which I would normally view as the middle person's anyway). So as a result of their decisions, none of us had a comfortable flight.
As a result, I find people doing this to be incredibly selfish.