So many restaurants I frequent don't take Amex, that I am pretty much used to it. I don't think any more or any less of a restaurant that does not take Amex.
If I eat at a restaurant, I'll eat there because I like the food, not because Amex isn't accepted.
That being said - I only have an Amex because it's my corp charge card. It's pretty easy to get re-imbursed for meals not charged on the Amex in that I don't have to send in any receipts for meals less than $75. IF I did have to send in receipts, I might choose to eat at another restaurant.
Hope this helps in making a decision.