should be enough time I think.
If I recall the drive from the airport into town is about 20min. You could get maybe 1 1/2hrs of siteseeing in.
Would be worth it to take a walk around the center and grab some snack/drink and relax if the weather is nice.