Nevertheless, the more often one tries, the easier the job gets. Even with my not too tiny hands (always XXL when purchasing gloves) it is possible within a few minutes.
But it still is a very stupid idea to construct a mechanism like this, even if the plane is outfitted with NC headphones, because there are lots of reasons why a pax could want to use his personal gear - quality is just one of these. Any other "new generation" C seat I experienced during the last months (Austrian, Thai, SAS) had a much easier way to use my own Bose QC2 (and not only LH is providing NC headphones!).
And one thing is definitely a hoax: you neither destroy anything of the original installation nor your own phones when you use your own - it is just inconvenient for yourself and maybe the FAs if you do not replug the original ones.