Amex won't call the lounge, the lounge won't call Amex - at least in my experience in LAS about 6 years ago (using Amex Plat ICC)
I was in a similar situation, however I was able to give the date & time that I had entered the LAS Centurion lounge a couple of weeks before, which the lounge agents were able to verify and they let me in
I knew my CC number and previous to getting access, I suggested that they could enter it manually, but they said they needed the physical card