When my job in Bulgaria ended, I had to turn in my Bulgarian residency permit (ID card). It was a formal withdrawal of the permit and I had to go to the immigration office and turn it in. I was concerned about what might happen when I left Bulgaria without the permit, but it was not an issue.
My residency permit here in Cyprus is not a card or a visa-like form in my passport; it is a piece of paper that I carry along with my passport. It is not associated with employment. I suppose if I ever left Cyprus, I would just let the permit expire naturally as there does not seem to be any requirement to turn it in.