What's the destination and are any special activities involved such as visits to places not normally open to the public or even factory tours?
For instance, for Russian visas, passport details are needed to arrange for a letter of invitation from a Russian citizen (usually done by a hotel employee or local travel agent). Some factory tours prohibit persons of certain nationalities. You might need passport or national ID details to obtain certain special tickets.
Do you trust this travel agent?