By the FA's logic, if anyone sits in an F seat, it's theirs and they don't need to show their boarding pass to prove it. Sounds like a terribly lazy FA and this is something to stand your ground on. It shouldn't really delay the flight for that passenger to stand up and walk back to their assigned seat. If they throw a hissy fit and refuse to move, that's when the FA/Pilot should call security to remove them.