I've become pretty immune to being worried in a car/other transportation. NYC taxis, drivers in Italy, India, Peru, etc - I act more as a spectator, being impressed by the show than scared by what may happen.
Walking the "Stairs of Death" at Huayna Picchu and such - if lots of tourists are doing it, and accidents only happen infrequently, I'm confident I'm more capable than the average tourist.
I suppose getting lost off trail in Nevada, solo, falling and fracturing my ankle, and having to hop, crawl, etc., back was a bit worrisome as the day was going to get hot. But I knew it couldn't have been more than a mile since I last saw a marker, so I couldn't be that far off.
The most scared/worried that comes to mind was waking in the middle of the night with our 11 mo. old daughter screaming, in a cabin about 30 minutes outside of Denali National Park. We had arrived late and they just left a key under the mat - no "front desk" at a bunch of cabins - cell phones didn't work, and I had no idea where any kind of services might be - we hadn't seen anything since Talkeetna 2+ hours back and hadn't ventured to the park entrance area to see what was there. It worked out OK, but that was one of the few times I was significantly concerned about a bad outcome.