As mentioned banks generally do not open accounts for those with a status of temporary visitor. These are from Citibank and Shinsei Bank (I imagine other banks have similar policies that might not be published online).

From Citibank Japan:
外国人登録証明書

在留資格が「短期滞在」、および「3ヶ月滞在以下」の方のお申込みは承ることができません。

Essentially, if your status is temporary visitor, opening an account cannot be done.

From Shinsei Bank:
"Your Alien Registration Card status is NOT “temporary visitor.”

From what I imagine is that people with certain statuses, eg officials (like gnaget's significant other) and diplomats, or even military personnel in Japan usually have ways to get accounts even though they don't have alien registration. Those are generally special circumstances where people might be able to pull strings or have agreements to work with.

However, I imagine someone with just a "temporary visitor" stamp in their passport, probably would not fall into any special category or have much luck in getting a bank account.