My brother lives on Jersey so I at least have a legitimate reason to visit the place. I've been very lucky that on my 50 or so visits to the island that the only time fog has affected me it was at LGW and forced a return to JER
When I was there last week it was just the day before that all the flights for the day were cancelled, and a couple of days after I left there were similar issues.
If you are financially prepared for it to belly up and have plenty of time for IRROPS then it shouldn't be a problem. There isn't a whole lot to see on the island, especially if the weather isn't good. On a nice day some parts of the island are rather pretty!