Tax exemption must be declared at the time of purchase, and the retailer must support it. In Japanese it's 免税 pronounced "man-zay". You then have to present your passport to the store clerk, and the clerk will complete a form and staple it to your passport. The consumption tax will be deducted at the point of sale. The form will be collected at or before the exit passport control, but I'm not exactly sure where.

FAQ on Tax Exemption Procedure