Changing physical currency should be a last resort. I always carry USD as a back-up, but I never use money changers. They pretty universally have really bad rates. Especially if you plan to exchange $500 CAD.

You'll be FAR better off using a credit card with no foreign transaction fees at all merchants who accept credit cards. For those who don't, I usually hit an ATM at HKG airport after landing (in the arrivals hall) and take out some cash $500-$1k HKD to start. ATMs are everywhere. I've had good luck with an ATM card from Capital One Bank (not my primary bank account) and Bank of China ATMs. Very quick and easy and the rate is better than anything you'll get from a money changer.