Your passport check is carried out at flight connections and is separate from immigration for customers ending their journey at LHR, fast track gets you through the flight connection lines quicker but can still be very busy.
Your longer connection is now ok.