I had it happen a couple times during a trip to Tokyo. There were a couple gay bars in Nishishinjuku that I experienced it as well. The guides to the gay bars actually indicate the ones that are welcoming to foreigners....and those were the places where people would try to chat with you, and they were really welcoming. The others simply did not allow foreigners. I figured it was because their patrons didn't want the social awkwardness and just wanted to relax.

A couple other times I was able to dine, but then its like an invisible force field around you where the seats stay empty around you even when its packed. Like how you usually can keep empty seats around you on the subway.

I have sat down and had diners at the table next to me move. But very awkwardly. With a lot of apologizing and nodding.

I am a non white American so I figured it was either xenophobia and/or racism or just odd Japanese awkwardness or something. It didn't bother me as much as I thought it would. I found it funny. Or sometimes I felt sorry for the person because my presence next to them spun them into a cycle of awkwardness that just seemed to ruin their day.

The great thing about Tokyo is that if one restaurant or bar doesn't want my money, there is an equally awesome one next door that does.