Community
Wiki Posts
Search

Bad Surveys

 
Thread Tools
 
Search this Thread
 
Old Jul 17, 2010 | 4:12 am
  #1  
Original Poster
 
Join Date: May 2003
Location: GEG
Programs: Motel 6 Club Avoir Le Cafard
Posts: 5,027
Bad Surveys

I'd like to see an outright ban on any surveys that consist only of radio buttons and do not provide reasonable means of free-form text input. Surveys of this type don't allow people to tell the stupid corporations that commission the surveys what it is we would really like to tell them.
mbstone is offline  
Old Jul 18, 2010 | 5:09 am
  #2  
FlyerTalk Evangelist
1M
60 Nights
50 Countries Visited
20 Years on Site
 
Join Date: Mar 2004
Location: SGF
Programs: AS, AA, UA, AGR S+, Choice Platinum
Posts: 23,317
Originally Posted by mbstone
I'd like to see an outright ban on any surveys that consist only of radio buttons and do not provide reasonable means of free-form text input. Surveys of this type don't allow people to tell the stupid corporations that commission the surveys what it is we would really like to tell them.
If you're referring to the student project that is currently being announced at the top of the forums, I doubt such a survey was commissioned by a company. It probably really is a student trying to gather data for a thesis.

Based my [admittedly very little] experience gathering data for empirical research papers, adding the ability to put free-form text adds a large degree of complexity to the research process and moves the poll from a measurement of hard data to subjective interpretation of what people meant.

A well designed poll should cover all of its respondents' points of view without leaving them feeling like they wished they could answer a different way. For example, in the project currently posted on FlyerTalk, I was satisfied with my answers to most of the questions, but questions 7 and 8 left me wanting--I wished there were an option for $0 (I wouldn't pay for a "health food & drink menu") or "I don't need this service" (I never use, much less would pay for, the ability to do airport check-in, except when I am required to by the airline's own backwards policies), and I think the spectrum of miles should realistically be wider (would an airline really offer an upgrade to business class for 1,000 miles? I wish they would, but 10,000 or more is much more realistic).

And, of course, the preamble to question 7 was misleading, given that Southwest is actually an odd duck among its low-fare companions in that it does not charge for most additonal services (opting instead to simply not offer them at all). I chalk that up to the researcher's UK background.

So, is this a poorly designed survey? Maybe; maybe not. Of course, the researcher is at a disadvantage, since he/she is subjecting himself/herself to scrutiny by a panel of several hundred thousand experts. The lesson to future researchers should be to, in order to save face and also get the most accurate data possible, perhaps do a test run with a few dozen people from the forums here to get input on the poll itself before releasing it to the general population.

But I don't think an outright ban is truly necessary.

Edit: There is some discussion of this particular topic in ORP: http://www.flyertalk.com/forum/only-...er-survey.html

Last edited by jackal; Jul 18, 2010 at 8:12 am Reason: Link to ORP
jackal is offline  


Contact Us - Archive - Advertising - Cookie Policy - Privacy Statement - Terms of Service -

This site is owned, operated, and maintained by MH Sub I, LLC dba Internet Brands. Copyright © 2026 MH Sub I, LLC dba Internet Brands. All rights reserved. Designated trademarks are the property of their respective owners.