If you are in MAD and leaving the Schengen zone, then allow time for the passport check. Last year we left the Sala Cibeles lounge and forgot that there was an additional passport check to access the gate areas (which are generally crowded), and it took over 20 min to get through.