I've never refused it if it was given to me for free, but there are occasions when I wish I had. Depending on which generation device is present, some of them emit audible feedback when near other electronic devices (such as phones) which can become annoying. Also, there's no way to cover it/stow it, so if you're not in a great neighborhood then it is usually visible to potential thieves (it's very tough to remove the device from the vehicle, but thieves don't know that). I try to bring a small black sack I got with a pair of headphones way back when with me if I know my car will have NeverLost so I can at least cover it up at night.
When I had NeverLost in a Crown Vic back in February, one of the cupholders was rendered completely useless
-J.