I think this is a great thing to do, and when I'm in Y it never bothers me, as the concept is fair and understandable. On most flights I have seen it, the FAs only hold back Y for a bit to let the F/J passengers who are ready to leave go. I would not expect/want the FAs to hold back Y for F/J passengers who are slow to leave.