I've never turned down anything in J/F yet but 95% of the time, don't bother in Y.
Might be because it's "fancier" food so different to me than a microwave style meal in Y.
If I eat in a lounge before a flight, I don't always want to eat on the plane anyway - maybe just a piece of fruit taken with me.
That said, next time I fly BA CW, I'll make sure to avoid any desserts. Found them like eating sugar coated with extra sugar and sweetened with more sugar.