...except in Japan. There, when I tried (having done the same thing you did, left my card in an ATM), Amex told me there was only one location: an obscure bank with only one branch in all of Japan. Furthermore, that branch was only open non-holiday weekdays and was on the opposite side of Tokyo from me.

But in most countries, yes, it should work.