I don't really worry about it. Not saying that I go around wearing a fanny pack but I'm not going to try to convince anyone that I'm a Canadian. I just think it's pointless—you're unlikely to fool anyone and if you're somewhere truly dangerous you're taking more precautions than a maple leaf patch on your bag.

Dress respectably and you'll be fine. Even if that includes jeans.

(That said, I normally avoid jeans because the material is heavy and doesn't breathe well but that's an orthogonal concern)