Well, we know that AA pilots taxi out on one engine to save fuel (their own initiative), but that shouldn't affect taxi speed.

Safety is a likelier reason. I've had several conversations with pilots and FAs who claim that the most dangerous time to be on an airplane is when it is on the ground, with all sorts of service vehicles, fuel trucks, and other aircraft in very close proximity.