I would not worry too much about that. Upgrade decisions can be made on board for various legitimate reasons...e.g. a passenger arriving late from a connection or a family separated and another passenger that refuses to move seats... or someone in Premium who wants to sit with their friend down the back.
The point is, if someone is going to be upgraded, I prefer it is me. And that is where a little friendliness can lead to good karma...