Sometimes, if the FA's won't properly do their jobs, it's time to take matters into your own hands...
Pull out your boarding card, ask the offending passenger to pull out his/hers, and explain how you're confused - but in a nice way, of course - how the other passenger's boarding card is imprinted with a seating assignment other than the seat in which they are now...