I can't imagine this happening. I've never had a passenger challenge me about my seat or when I board or anything like that.

I did, however, get challenged by an FA in international C a couple of weeks ago. Right after take-off I discovered that my seat was broken. The FA apologized and had me move up to another seat several rows forward. About 10 minutes later the purser strode up to me and demanded my boarding pass stub, telling me that I needed to return to my seat immediately and basically lecturing me about the difference between the three cabins on the aircraft. I couldn't get a word in edgwise, so I handed her my boarding pass stub. She saw I had a seat in the same cabin and then was puzzled enough to let me explain. To her credit, she immediately apologized profusely and seemed genuinely embarassed about the whole thing. She ended up being really great the rest of the flight.

But a passenger? I don't think I'd hesitate to tell him or her off.