I'll start by saying I don't have kids.
Panama is very safe and accessible. Of course, like anywhere, probably even where you live, there are places you'll want to avoid (such as Colon or certain parts of Panama City), but you shouldn't expect any problems in Panama. It's a stable country with good infrastructure. I'm sure your kids would enjoy the canal, especially one of the boat tours where you'll see various monkeys, sloths, and other wildlife. And the San Blas Islands might be fun. There are a few bigger beach resorts in Panama as well, but probably not as many as you might expect.
Having been to both a few times, my impression is that the tourism industry in Panama is not nearly as mature as somewhere like Costa Rica, and not geared as much toward kids. I'm not saying don't consider it, because it's a fantastic country. But do your research for the areas you're interested in and make sure you can find the types of hotels your family needs, and that you'll have access the tourism amenities you might be used to -- for example, there are some fantastic beaches in Panama but don't expect them all to be built up with restaurants and places renting surf boards and snorkeling gear.