From BUF parking to my home in North end of Toronto is about 80 minutes driving at the flow of the traffic. CBSA border varies from 5-20 minute for me most of the time.
There are times that the wait is over 90 minutes during the border crossing rush hours.

5 hours driving from BUF I am almost at DTW.

If you are flying into BUF plan to make the drive through wine/fruit country enjoyable. Niagara Falls, although overly commercialized still has the falls. There is also quaint little town off Route 55 from QEW call Niagara on the Lake which is wonderful for a day trip.

The BUF to downtown bus is a low cost way of getting to Toronto downtown if you can deal with your luggage. It never worked for me for I do not live downtown and the bus schedule is not great with only 3 buses/day.