I experienced a similar situation to the OP's on a recent SFO-SYD flight. Many passengers seemed frustrated, but as others mentioned, it is also good for circulation to take the stairs to the other lav. Each time I went down, it was available and I was greeted with big, genuine smiles from the other F/As. On the return SYD-LAX, it was not a problem.