There is priority security at YYZ but that won't help you getting through US CBP. Everybody is equal there.
I have never experienced any problems in the afternoon, usually it is rammed in the mornings because they only open relatively shortly before the first bank of flights and then - surprise, surprise! - you get long queues.
My experience is that there is no point in turning up more than 2h in advance - if there is a massive queue they will not let you even join it until about 1-1.5h before the flight anyway!