You should allow at least 2-3 hours to be back at the airport for your onward flight. So literally, you would only have 3-4 hours to spare. You must also take consideration the need to come and go which can range from 30-45 min depending on how traffic is on Granville (the main road linking the city centre and the airport). Speaking of which, your best bet in getting to the downtown areaa would be by cab which is approximately $25 CDN each way. And no, you don't need to pre-book your taxi on the way to town but you will need to call in advance for the return trip to the airport.