The airline pilot in me can see how the FA would be annoyed (most hate serving, especially alcohol, and honestly have gotten quite lazy, even for premium passengers).

However The premium traveler in me feels you didn’t do anything wrong. Perhaps a quick glance to the rear of the plane to see if the carts were out and to wait for them to finish, but regardless the FA should not have reacted like that, and especially shouldn’t have told you not to use the button.

I would file a complaint.