Agreed, I wouldn't worry about taking time in the lav. If it makes you feel better, tell a flight attendant. I don't think it'll matter to them one way or the other.

I think I would only feel comfortable pumping in my seat if my family occupied the whole row on that side of the aisle, and I was in the window seat, with a boob tent. Fortunately, I don't think anyone would hear the pump over the plane's noise.