A "down at the heels" area in Japan is much less dangerous than a "down at the heels" area in most other countries.

I once inadvertently booked a hotel in an area which a local resident later told me was a red light district. I went with some trepidation, but there were only a few signs of the area's main industry: a store selling sexy lingerie and S&M equipment down the street, a sign in the hotel lobby telling prostitutes that they and their customers were not welcome, and the lack of any restaurant options other than a McDonald's and a few late-night noodle shops. There were more than the usual number of love hotels, but since you can find love hotels just about anywhere in Japan, that was no big deal.

The only time any place in Japan made me nervous was in my student days, when a Japanese friend took me on a barhopping and coffee shop hopping tour of Shinjuku. We went into one basement establishment where the customers were all kind of ragged looking and obviously zoned out on something. Knowing about Japan's harsh anti-drug laws, I told my companion, "I don't want to be here as a foreigner if the police raid this place." My friend understood this concern, and we left immediately.