Even when someone reclines during the meal service (which I think is rude), I still don't use that a reason to push, kick or otherwise mess with the seat in front of me. I really don't think you were out of line....but he was.