Still, I think a lender not knowing the address of your residence could make them antsy.
Why, if they don't ask? Like I said, one or two banks did ask for my physical address, and I provided it. (The one I remember for sure was US Bank.)
But if a bank only asks for my mailing address, it's their fault that they didn't ask if my physical address is different from my mailing address. I'm not withholding that information, most banks simply don't ask for it.
I'm not the only person whose mailing address is not the same as their physical address. I've run into people on FT who use a mailing address for a service that forwards their mail around as they travel, because they're away from home for much longer stretches than I am.
So banks (especially ones issuing travel cards) should anticipate this if they really care.