Did they let you through in the end? Did you show them your inbound boarding pass?
Priority boarding is a bit hit and miss, I remember someone trying to turn me back at AMS once because while having a gold card, I wasn't in CE. I am sure they make up the rules as they go