Often, the club/lounge area is accessed via your key card so it is programmed at check in and regulated that way.

It is typically a pretty relaxed situation, however at larger, busier properties such as the Grand Hyatt in NYC, you need to present your key in the elevator to access the club level floor, present your key to access the lounge, and there is also sometimes an attendant at the entrance who may or may not ask for your room number.