You do realize that is exactly what they are for: Check arriving passengers. I'm not going into the merits as I don't know how they behaved and if you did something wrong (e.g. filling out the form improperly).

I've been giving a hard time by immigration officers and customs officers. If I stopped traveling to those countries, I wouldn't be traveling much these days.

Maybe you just looked guilty. A family member manages to get stopped every single time by customs. Maybe you were just the 1 in X00 that gets selected for more thorough screening. Maybe the IO just had a bad day. Knowing the Japanese, I doubt that any of this happened on purpose.