Part of the problem is their are two lounges that say "ANA" on them. One is strictly an El Al lounge (on 4th floor?). The other one should let you in (if they are open when you want to fly). I have been in the Varig, SQ and ANA lounges no problems other than timing.
Varig/SQ closes at 10AM and then reopens later.