I’ve faced this too. They “discourage” people from using the shuttle for lounge access. It’s a stupid policy. Also, they do check boarding passes if you say you’re going to B.

So always say you’re going to A/C and then ask the shuttle driver to drop you at B. It’s a scheduled stop.