Something here just doesn't add up. The whole point of Gold Choice is that you can swap without seeing anyone, and without having any reason.
Are you sure the car you picked was actually in the Gold Choice area, and not (say) in a normal spot next to Gold Choice? (Sometimes it's not clear where Gold Choice ends and the normal "#1 Club Gold" spots start).
How did the manager know to call the gate agent whilst you were still there?