Even with the ridiculous tourist add-ons - I would get a car AND driver.
Take a taxi into town and get the car/driver as needed.
The local daily rates (with driver) available to Sri Lankans are less than your driverless internet rate.
Happy wandering.
Fred