I think this depends on what the restaurant actually does. I've seen them come over, no decanter, uncork the bottle, pour into regular glasses without decanting, and leave the bottle for me to pour seconds. In that case, the $20 is a penalty fee for bringing my own bottle.

If they provide a proper decanting service, good stemware, etc, then the fee is for a service and perhaps reasonable.