Perhaps not what you want to hear but there are times you need to decide whether your purpose is photography or vacationing.

I don't mean the above to be snarky at all. The way I deal with your problem is to leave the camera bag in the hotel room or rental car trunk when I might "go swimming in the ocean" or anything similar. I always have a decent pocket camera with me in addition to the "big" camera. For the occasions when I leave the K5+gear behind, the little camera is still with me. I wouldn't feel so nervous about leaving it wrapped in a towel or in a shoe while swimming but can still get decent travel snapshots with it. When I want images better than "snaps," I simply concentrate on photography and forego the swimming.

At the end of the day, some of my better travel pictures, and a few quite good ones, have been taken with the little P&S.