As far as YYZ is from downtown Toronto, with 6 to 7 hours to kill, I would definately head downtown. That's far too long to just hang around the airport, and there's really nothing around the airport worth visiting, so a few hours downtown is a great choice. If you still have a little time after the HHOF, there's lots to see just wandering around in the immediate area.