Traffic concerns.

Not just food places, but cleaners and drug stores for example get denied because the planners feel they are in a "mixed use" area, where foot traffic is important, and a drive thru does not promote that.

Increased noise to the neighbors, things like that.

Cases have gone as far as state supreme courts when they were denied (Dunkin doughnuts against the town of Hempstead NY for example)