Interesting. A nice punch in the nose would do the trick, but then you get charged with battery.
Perhaps seek witnesses around you to complain as well--behavior like his was obviously annoying to more people than just yourself. One squeaky wheel can squeak, but 5 or 6 of them will definitely get the grease.
As a final measure, his behavior was clearly harassment. I would approach the FA and say, "I know there's not much you can do about this, but I would like to press charges against this passenger once we're on the ground. Can you be sure the Captain requests the authorities to be present at our destination?" (or something to that effect)