I kind of gave up on buying cloth for myself in Japan, I am 6 ft. tall, 190 lbs., and shoe size 11. Even “Extra Large” size clothing in Japan to be tight around shoulder for me. I was also directed to Isetan previously. After XL clothing it will be XXL and XXL but found out that those are meant for “person of size” and I will have enough space around belly to fit few soccer balls but still shoulder area to be tight.

As for souvenir T-shirts Asakusa Sensou-ji area and Narita and Handeda international terminal are good place for size to fit Western people.

I once met an expatriate living in Japan and he was planning a vacation to Guam, one of the reason choosing Guam was so that he can do cloth shopping.

But again, I gave up on cloth shopping for myself in Japan so I have not actually looked around recently.