In econo, I _always_ turn around and make eye contact with the person behind me before I lower the seat. I raise my eyebrows and give a half-smile to say, "sorry". It eliminates any guilt I have over doing it. If they are uhm, shall we say, large, I will suffer and leave it up.
The only problem I have with those in front of me doing it, as the cat says, is when it comes flying back at me like a spring-loaded mouse trap.
The Golden Rule definately applies here.