I was in J with a family member in Y who needed an iphone cable, so I asked if I can go down the plane to hand it over.
They said "it's ok for you to go down there, but they cannot come up"

Others have mentioned elsewhere you can swap at the start of the flight, but you can't swap or keep swapping after that mid-flight.