Yeah, so the checkin employee at ORLY airport wasn't particularly talkative, but she mentioned that "this is normal because you have a visa".
So it seems that anything but ESTA does indeed cause them problems. Not quite sure how it works for some people though

They then spent another little while trying to convince me that my 30 kilo luggage is too much.
I just kept showing them the BA website mentioning 32 kilos for biz bed customers and told them that this wasn't a problem flying from the US to Europe. Then they changed the stance that 32 kilos is "the maximum", but 23 kilos would be better.
After about 10 minutes of back and forth they just said ok and checked the bag.

On the way over we had our flight canceled in the late evening prior and were rebooked from JFK instead of EWR.

This was my first time flying BA and I think I'll try to avoid it in the future :-/

(End of rant, thanks for listening)