I usually return them to the desk or leave them on the desk in the room but I've never heard of a hotel caring too much either way about the credit card type. When the plastic punch keys first came out they were much more diligent about collecting those.

To me it is more a question of recycling. I prefer to recycle whenever possible, but there are no guarantees, the hotel/maid may just through them out too.