It's nowhere near as demanding as off-road driving. The road to Crystal Mill, Colorado, as an example, requires a high-clearance vehicle because of rocks, occasional deep water, etc. The off-road paths to the natural pools on Aruba require 4 wheel drive. On a scale of 1 to 10, if Crystal Mill is a 7 and Aruba is a 9 (and 5th Avenue is a 1), this rings in around a 4. Locals in 1990s Ford sedans drive the road no problem -- as long as there's no active flooding.

All that said, highly recommend keeping clockwise throughout, it's a much calmer experience. Because there are so many people who drive both segments of it clockwise, you'll constantly be passing people if you swim upstream, whereas on our drive southbound, we only encountered a handful of vehicles driving against us.