Actually, it's quite easy, at least it was during my visit. I just rode my bicycle across the border. No government = no border controls. Then again, if you hit border patrols on exit (for instance at an airport) and don't have an entry recordation in your passport...

I've only had real trouble with three countries (out of about 150): Cuba, Libya and North Korea. I do try to adhere to my country's (United States) limitations on travel (hence the Cuba and North Korea challenges). The other problem was Libya. This was 10 years ago, and they just fought me to exhaustion on independent travel to Libya. This has probably changed.

Otherwise, I've gotten visas pretty readily, though prices are often on the edge of extortion.