Yeah, very annoying. My long-used 100% solution is as soon as you feel his hand on your seatback, lean ever so slightly forward and lightly push the "recline button" and your seatback will be pulled back unexpectidly onto him. They usually get the message the first time it happens, and I have never had to do it more than twice on a flight.

MisterNice