Indeed, fast track isn't synonym of fast exit. I once saw more people in Fast track than in normal queue !
If you go often to Dubai, you should consider applying for e-gate, if your nationality permits. It comes with a cost, but passport control is done within seconds (when the machine works).