You want to avoid backpackers with Lonely Planets in one hand and cigarettes in the other so you avoid MP but want to go to Arequipa/Colca? Chevre.

Since your stated criteria is to avoid places packed with smoking gringos, then the advice upthread about going north is spot on. Trujillo with huancas, Chiclayo-Lambayeque with Sipan, pyramids, Cajamarca with Kuelap fort, Gotga falls, even Ayabaca with Aypate the Machu Pichu of the north, northern beaches. For Frommers toting tourists (remember Peru is a tourist destination for Peruvians too, but they don't need Lonely Planet) then think that MP has 4000 tourists a day, Kuelap fort 4000 a year and Ayapate inca site 4 per week. Of course, in terms of impressiveness, MP is most impressive, Kuelap less impressive and Ayapte even less impressive. So the more impressive the site, the more toursists with Lonely Planets in one hand and cigarettes in the other. Funny how it works that way.

By the way, you missed something when you missed Cartagena, it is special and worth visiting.
.