For commercial: I'd prefer to see them in uniform, including a coat, at least for the beginning of the flight. What happens during is up to them -- at that point I'd rather they be comfortable. Afterwards, I'm fine with dress shirt & tie, with or without coat.
Does it signify skill? No. But it does give a certain amount of respect to the position, which I'd hope wouldn't be lost on the flight crew or the pax.
For charter flights (general aviation), I'm not too picky, especially if it's not the first time I've flown with the pilot. I've been on many charter flights where the pilot was wearing a T-shirt and cut-offs. I DO prefer the pilot leaves their shirt on, even on a Cessna 172. For those unaware, most (all?) Cessna 172's lack air conditioning. I know the '172 can be miserable when waiting to taxi in a Florida summer, but please, please, please, leave the shirt on.