The rubber covering is completely pulled back, right? Sometimes the rubber can hold the clip in place even though you think it's down all the way.

You might actually push it slightly back in and then press down on the plastic clip again and pull.