Do everyone a favor and report the FA.
My policy is NEVER to move seats for any reason without a new boarding pass being issued. You should have asked for downgrade compensation on the spot since you were moved to E- and demanded the FA get the GA to issue you the downgrade compensation.
I would also report the person the took your original seat. If they were a pass rider, it will cause the employee trouble that their guests were causing trouble.