I had this last week. I live in Switzerland and have both Swiss and UK passports. The rule is use UK passport in the UK and Swiss passport to enter Schengen, but I had entered my UK passport on the booking. No problem arriving in LHR and for once even the e-gates worked. Trying to return to GVA, I checked in on-line, had my BP in the app, but on trying to drop my bag, got a "call agent" notification. The agent who came was a little stumped for a few seconds with much hammering of keyboard, but finally announced that the system was unhappy with me having a UK passport and being resident in Switzerland. She wanted to see a resident's permit, which of course I don't have. I showed her my Swiss passport, followed by much more hammering at the keyboard, as she presumably tried to override the "computer says no" message.
I wonder what would have happened had I been HBO, as I was already checked in and wouldn't have seen anyone before boarding at the gate. Would I have got the dreaded double-beep?
Next time I have to do this, I will try and switch the passport in my booking after having travelled to the UK.