That's a quad band phone so it will work anywhere in the world IF you have international calling activated on it. You will have to call AT&T and have them turn it on.

There is an optional international calling plan (AT&T World Traveler) that costs $5.99/month and reduces your per-minute international roaming rates. Roaming rates in Japan are $2.29/minute but with the calling plan you'd pay $1.69 a minute. The World Traveler plan requires a 1-year commitment, which they aren't good about telling you.

Buying a SIM card on arrival in Japan would almost certainly save you money. Pop out the AT&T card and put it in a safe place. Replace it with the local SIM and make calls. Simple as that. The new SIM card will give your phone a new number, however, so if you regularly receive calls you may want to leave a message on your AT&T voicemail directing people to call the Japanese number associated with the new SIM.

Oh, I forgot to mention, you can only swap out the SIM card if your phone is "unlocked." AT&T will usually unlock your phone if you are at least a year into your contract with them. Otherwise you're SOL unless you're willing to jump through a bunch of hoops to get it unlocked.