I believe the OP is correct in assuming no access. BA's lounge access policies are far more generous than those laid down by oneworld, and it's only that generosity that allows access to BA lounges off the back of an incoming boarding pass.

I don't believe IB mirrors these rules, in spite of the IAG connection, so the oneworld requirements would apply - so no access because the MAD-LHR flight is in Y.