jamiel: A friend of mine tried to see Cuzco on the weekend of a two week business trip to Lima. He ended up spending the whole weekend in bed with altitude sickness. Cuzco is way up there. You might want to check with your group if a side trip is worth it without having a couple of days to adjust to the altitude.
OTOH, not everyone is prone to this problem. I had no difficulties making a quick adjustment in BOG, which lies at about 8,000 feet up.