Rent a car. Bahrain is easy to get around, and you can see the whole place in a day or two. If you are just planning to stay at the hotel, than taxi is indeed the best way.

And as far as your concerns - I'd be 1000x more scared visiting LA than Bahrain!