Just as a datapoint:

I flew MAD - AMS earlier this month as OWS (non-IB, AY) and got Fast Track from MAD to AMS. However, I did not get it from AMS to MAD.

The boarding pass indicated Priority security.