For my case, it has been worth it all the time. Usually hotel branded credit cards gives out free night awards certificate yearly that can be worth beyond the annual fee. So, I just think of the card as no annual fee card.
Also, the points you earn during sign up and using the card can give you nice vacation without worrying the high hotel room cost. For example, Marriott Bonvoy, if you use points for 4night you get 5th night free which I love the most about the program.