Definitely leave the airport - you are a short taxi ride away from lots of nice places. You only need to be back at the airport 30-40 mins prior to your onward flight. St Brelade's is lovely; another option would be a taxi to El Ticos in St Ouen's Bay (also 5 mins away, and has spectacular views out to the Atlantic) for an early lunch / brunch and then a walk along the bay. Ask the taxi driver to come and pick you up for the return to the airport. If it's raining, I would still head for St Brelade's - have a coffee and read the papers in the coffee lounge at L'Horizon hotel, and then the Oyster Box for an early lunch.
Thanks a lot, this looks like a great option also! Will hopefully get to do one of the two, will make a good break in a long day (GCI-JER-MAN-ABZ)!