I notice it hasn't been mentioned but recently the express highway has opened from the airport and (for a fee) saves so much time getting to Colombo than the traditional main road. Ive not personally tried it yet (heading back next month) but I know from friends and family their 2.5 hour trip from their house to the airport is now 40minutes.
I'd head to Negombo, get a day pass at the Jetwing Beach (1000rupees last time I reckon). Lay by the pool, dip your feet in the sea and relax.
Clearing immigration is usually fairly quick, as long as their aren't too many planes which have arrived just before you had. Luggage never takes very long.