I carry a $1,000 SLR with a very substantial $1,200 lens on it and and smaller lens in my bag and I'm rarely worried. My camera and I have been to about 6 or 7 countries together just this year. I carry a Lowepro Slingbag and it's very secure. I can loop the zipper pull through one of the clips and it can't be undone without me easily noticing.

You'll see countless people carrying cameras. Just be aware of the camera, don't be careless and you'll have no problems.