Just be sure you understand you are qualifying as a United Club member, not as a Visa Infinite card holder. The lounges that will let you in will be looking for the digital (or physical) United Club card as the qualifying credential. They won't care one bit about the Visa card that got you United Club membership.