I was on Capri back in 1989. At that time, we were staying in Sorrento and took a day trip there.
You can go by "slow" ferry or "fast" hydro-foil. See the blue grotto; touristy; but definitely cool.
Naples or Sorrento would be the starting point.
- HobokenFlyer