I like Consumer Reports - I'm a long time subscriber - but I find their reports are often targeted towards a middle of the road audience. Maybe with lawnmowers and refrigerators that's a good thing, but in the case of airline ratings, it's not very useful to this frequent traveller.
For instance, in the story on airline ratings, Consumer's does not incorporate frequent flyer programs into its quality ratings. While mileage programs may not have an impact on the folks who fly once or twice a year, they greatly impact my decision which airline to fly. My switch from UA to CO a couple of years ago means I now fly first class on nearly every flight without having to pay for upgrade coupons or wait at the gate to find out if my UA upgrade waitlist cleared. And my miles don't expire anymore. Both are huge plusses for CO; considering I would spend equivalent amounts on either airline, CO delivers much better value for my money.
Also missing from their ratings categories are convenience of schedule and frequency of service. Maybe Aloha's a great airline (they were top rated in the story), but can it get me where I want to go? What about the three stops I have to make on SW (the third rated carrier) to get me from here to there?
Consumer's did a survey of options for obtaining frequent flyer points about a year ago and panned mileage cards. True - they're a poor value for the average consumer charging 10K a year or less. But what about the business traveller like me who charges upwards of 70K per year? In that case, they're a good value, since I'm using a credit card anyway. But the Consumer's article never mentions that a mileage card might be a good option for some consumers.
I doubt if many people on these boards will find the story in Consumer's terribly useful in making a choice of carrier. It's interesting reading, but too superficial an analysis for my taste.