Unfortunately, I don't think just being loud and smelling from booze is something that should trigger action on the part of the FA.
If he is using inappropriate language, hitting your seat back repeatedly, being aggressive somehow, etc., then it crosses the line.
I believe the flight attendant was correct.