I am always in jeans and a long sleeved shirt. While there are times I've got my suit on still, its rare since I started going to SkyClubs.
I don't really care what the GA thinks to be honest. I go in and if they give me a hard time, so what? I'm almost always the youngest person in F by far (unless infants are in F), but i often fly the same route for 6-9 months at a time and end up seeing the same GA's after a while. They are usually nicer the third or fourth time around, but still... it does happen. But I just shrug my shoulders and move on.
If it ever gets bad, I guess I can move on elsewhere, but to date, its never bothered me too much. More often than not, the GA's are friendly enough and don't give me a hard time.