Count me in as one who carries it with me whenever I'm off the ship. I keep a copy in the safe and another in the luggage, but the original goes with me. I try to take care to not wear backpacks or carry a purse (usually a dreaded fanny pack, carried around front) that could be easily swiped or pickpocketed. My tourguide in Florence almost had her backpack pickpocketed in front of the Duomo -- very crowded place and she was distracted.

I just can't predict when there may be a medical emergency or other mishap that might detain me or Mr. Xine when we are traveling, and I want to make sure we are prepared in case we do miss the ship.

Do you also carry photocopies of your money, and keep the real stuff in the safe?? You know, just in case it gets stolen or lost? AFAIK, an Embassy won't replace that... so I just put the two together (the money and the passport) and keep a careful eye on them.