I would have held it. I usually travel alone and prefer the window seat but I rarely use the restroom. On long flights I will get up with another person in the row gets up and figure it might be the only chance the stretch my legs.

However, if I was one of the people sleeping I would have completely understood if I was awakened to someone in my row needing to use the bathroom. Especially if it was a long flight.