I always purchase travel insurance for my trips, primarily for the medical component as many countries require proof of ability to pay before treatment, and of course US health insurance is not accepted. There are two important pieces to the medical coverage. One, you want to make sure you purchase the policy within 14 days or your initial trip deposit so that you can get the pre-existing condition waiver. This rule varies among companies, for some it is 21 days, but 14 days is a good rule of thumb. Two, make sure the medical coverage is first payer, meaning that your travel insurance policy will pay for your treatment first, and then go after any other coverage that you may have for reimbursement later.
I also purchase a separate annual medivac plan.
MikeFromTokyo is spot on that insurance is about figuring our your personal tolerance for risk. Some people may prefer to self-insure, meaning it would not impact their lives in any significant way to shell out $100,000 for a medivac, or lose $80,000 on nonrefundable trip components.
That being said, a lot of the potential reasons for claims are not things that most people think could really happen to them - volcanoes, airport power outages, hurricanes in New York, getting hit by a car, tripping and breaking your arm, supplier bankruptcy, etc. Many people will say, well it's summer so so our flights should be on time, and grandma is healthy, so I don't need insurance.
I have a low risk tolerance, in part due to being a TA and seeing all of the things that can and do go wrong. Plus, my family's business is insurance, although not travel related.