Just so it is clear in my mind are you saying the airline agent actually said something along the lines of “I deliberately gave your passport to another person.” and not something like “I accidentally gave your passport to another passenger.”? Since they were able to call the person who had your passport back to the counter it sounds like they still had the other person’s passport or ID and had made a mistake. While I share your sense of distress I’ll have to say of all my IDs the one I’d be least worried about would be my passport. Here in the US passport information is rarely used in ID theft as there is relatively little information on the passport and at least I have never been asked for my passport to open a bank account, get a loan, or get a credit card.