Never been a problem for me. I think it's a cultural thing too - outside the US it generally isn't done.

I never once was told taking home leftovers was low class - and my mom and grandma were/are the supreme arbiters of all that is Emily Post.