In 30+ entries to Japan in the past few years, I have never had my bags searched as a 20-something white male (fluent in Japanese, which may make a difference). However, I would absolutely not attempt what you are proposing for two reasons:
1. There are random (or not-so-random) searches. I have always seen at least a few people being searched whenever I go through customs. There are routinely roaming drugs dogs. If you are searched, they will have you both on making a false declaration and importing an illegal substance. Drugs offenses are not treated lightly in Japan, and at the very least, you would be detained for some time and deported with a ban on returning to Japan; at worst, you could face many years in prison. Quite simply, you absolutely do not want to get involved with the Japanese justice system, which has a near-perfect conviction rate, harsh penalties, and a dismal human rights record.
2. Because you know you are doing something illegal, you are likely to exhibit signs of nervousness as you go through customs. The officers are trained to pick up on this, and the fact that you are nervous will make you much, much more likely to be searched than anyone reporting their experiences here (where they are not knowingly doing anything illegal). And when this happens, see point number one above.
I would strongly advise against what you are suggesting, and would recommend pursuing an application for a 薬監証明書 (characters above were incorrect, FYI in case you try to go Googling) or looking into potential options for getting a local prescription after arrival as Nogbad suggests.