Let your hotel organize a driver for you. They have an interest in not pissing you off too much, and they're likely to have access to more candidates than you will.

There's never a shortage of drivers in Bali so availability isn't an issue. It's more a matter of, as you intuited, not being endlessly pestered with unwanted gift-shop stops.