Ha, this makes me laugh - I've never had luck on that route. Last time I tried to return from Ghent to Lille, I ended up getting stuck in Kortrijk and having to take a taxi back to Moeskroen and a bus over the border (having missed the last connection of the day). At least it's comforting to know it's not just my bad luck...
(It's also where I was told off for trying to use 'Rijsel', rather than Lille, when attempting to speak Dutch. Ah, linguistic politics...)