Yes, be cautious. A few years after 9/11 I was on the US side of Niagara. A lady told me that my drivers license was enough to get to the other side. So I tried my luck and was denied entry. They took my information and put it in their records. I have always been sent to detention when transiting Canada. That was while I was an Indian citizen. Then I became a US citizen and the drama still continues. Don't know how they tracked me down. But since OP has used his Canadian passport without problems, I think he is okay.