Most of my op-ups happen at the gate. One time however, I was already sitting in my Y seat (on JFK-ZRH) when a ground agent came to me and asked me to "move up". And just as mbr has pointed out, other passengers started inquiring and muttering about being treated "unfairly".