EP/SEs do have access to all lounges at all times, with the exception of trans-border at YVR, YYZ, and YUL, LHR, and possibly YOW.

Even YOW, I have been able to get "past the problem" but with some difficulty.

If TC has a problem, it is not a universal one. I have no problem at YYC. At one time, all I needed to do was show my SE card to security in YYC but I need a gate pass now which I have never had a problem getting from the AC agents.

For the international lounges, it may be that it has to do with the arrangment related to US immigrations which are often (always?) in adjacent parts of the terminal? Who knows--sometimes there is no rational reason for these decisions.