Are Airbnb reviews reliable?

Old Jul 1, 16, 6:43 am
  #1  
Original Poster
 
Join Date: Sep 2006
Location: Jersey City, NJ
Posts: 43
Are Airbnb reviews reliable?

Are they reliable? I think not, but I'm not exactly experienced. After having my review deleted, a 5 star review mind you, I join other cynics about the Airbnb review process. There's simply too many positive reviews and it seems statistically improbable.

My recent Airbnb experience was my first. There were some minor issues with the apartment and one not so minor. I wrote about these issues. However, the review was mostly positive and I decided to give the apartment the top rating - 5 stars, mostly because the owner seemed to go out of her way to fix things. In hindsight, it didn't deserve 5 stars.

The review was posted for about a week or so. I went to look to see if the owner commented on my review and it was gone. I couldn't figure out why. A call to Airbnb gave me the answer. The owner complained about my review and Airbnb removed it without even checking the review (as per the CSR I spoke with). He looked it over and said there was nothing wrong with it and agreed to put it back up.

In the end, Airbnb is doing the right thing. But I have to wonder how often this sort of thing happens. If it is with any frequency, that could be the answer to why there are so many positive reviews.

There was a more recent review posted about the apartment I stayed at. It was filled with laudatory exaggerations. For example, the reviewer stated that the ocean was so close that he/she could hear it. Baloney!! It was about .25 miles away. And there were no waves. Even when you're on the beach there's nothing to hear.

Another review stated that it slept 5 comfortably. Baloney!! There's only 1 queen size bed. The living room has a cheap and uncomfortable sofa. No way.

It makes me wonder.
gsusser is offline  
Old Jul 1, 16, 6:58 am
  #2  
Moderator: UK and Ireland & Europe
 
Join Date: Jan 2003
Location: Biggleswade
Programs: SK*G, Lots of Blue Elsewhere
Posts: 13,362
It suffers from the "everything is 5* unless there's something seriously wrong" syndrome that affects so many reviews. You have to read into the comments a bit more (rather like with TA).

I know perspectives differ, but really, not everything deserves top marks.

Stayed at an apartment recently - good location, nice apartment, just as described on the site. However, the key wasn't left where the instructions said (couldn't initially find it), and the owner wasn't available on the phone (over quite some period, despite us saying in advance when we'd be there). Once we were in, we found the apartment in a good state, but the kitchen (and in particular the fridge) hadn't been cleaned especially well. We went out to buy equipment to do so.

None of this was major stuff, but does it deserve less than 5* as a result? Particularly when other places we've stayed, the owners have gone out of their way to accommodate us.

It's hard to calibrate... So you need to read into the comments, I think.
stut is offline  
Old Jul 1, 16, 7:20 am
  #3  
 
Join Date: Sep 2015
Location: London
Posts: 517
AirBnB claims that it doesn't delete reviews unless it violates T&Cs... could it be that the host "reported" your review for "violating the guidelines" and AirBnB automatically hides the review whilst it's undergoing review?

I don't think they are censored as I've seen plenty of bad reviews, but I'm perhaps too trusting

Originally Posted by stut View Post
It suffers from the "everything is 5* unless there's something seriously wrong" syndrome that affects so many reviews. You have to read into the comments a bit more (rather like with TA).
Agreed that this is more likely to be the issue. On AirBnB, "does roughly what it says on the tin" seems to be calibrated at 5* though I'd disagree on TA, where that seems to be calibrated at 3 or 4*. Uber is another example - anecdotally, I've never heard of anyone give anything but 1 or 5*s!

The problem is, this star inflation makes it hard to single out the truly exceptional. (Actually, don't we have the same conversation about high school grades in the UK every year? ) Hence on AirBnB and TA, I'll avoid anything below a certain rating and a very high rating may sway me, but I'll normally make my choice by looking at the comments.
SwissBritMiss is offline  
Old Jul 1, 16, 7:46 am
  #4  
Original Poster
 
Join Date: Sep 2006
Location: Jersey City, NJ
Posts: 43
Stut... yes, I am guilty of the over-inflated review rating. Good food for thought.

Originally Posted by SwissBritMiss View Post
AirBnB claims that it doesn't delete reviews unless it violates T&Cs... could it be that the host "reported" your review for "violating the guidelines" and AirBnB automatically hides the review whilst it's undergoing review?

I don't think they are censored as I've seen plenty of bad reviews, but I'm perhaps too trusting
Yes, you are correct that the review was hidden though I have no idea why the host reported it. I don't know the Airbnb review process and I did not know what "hidden" implied. The CSR made no mention that it would eventually undergo review. On the contrary, he stated it was an Airbnb oversight.

The exaggerated reviews are not the fault of Airbnb unless one wants to get really cynical. However, it makes me wonder if one can rely on consumer reviews, though I suppose this applies universally, i.e., Amazon, eBay, TA, etc. Then again, the apartment only had 4 reviews (all 5 star) with only positive comments when I booked. Four reviews are not statistically reliable and I should have realized that.
gsusser is offline  
Old Jul 1, 16, 8:57 am
  #5  
FlyerTalk Evangelist
 
Join Date: Apr 2000
Location: FLL -> Where The Boyars Are
Programs: AA EXP 1.7 M, Hilton Gold, Hertz 5*, AARP Sophomore, 14-time Croix de Candlestick
Posts: 18,669
Regarding public reviews, in my opinion it is OK to include them as a part of your decision making, but only a part. Common sense and caution need to be in the mix as well.
trooper likes this.
Non-NonRev is offline  
Old Jul 1, 16, 9:55 am
  #6  
 
Join Date: Apr 2015
Location: Riberas del Pilar, Mexico
Posts: 364
Airbnb is better than Homeaway is. There is NO support for the renters from that site & they will NOT allow a less than stellar review. I'd use Airbnb over Homeaway any day.
Gringa is offline  
Old Jul 1, 16, 10:51 am
  #7  
FlyerTalk Evangelist
 
Join Date: Nov 2004
Location: ORD
Programs: UA 1K
Posts: 15,862
Airbnb reviews are only published after both the host and guest have reviewed each other. This probably inflates the rating.

If I had a bad experience it's likely the host knows it and by not reviewing me they avoid my review going public.
milepig is offline  
Old Jul 1, 16, 10:56 am
  #8  
 
Join Date: May 2005
Location: Cincinnati, OH
Programs: Delta, Starwood, Hilton
Posts: 438
I rarely look to any review for being positive or not since that can be a very subjective opinion. Instead I read reviews very carefully indeed to find out the most I can about any rental. I check to make sure that the owner listed is the owner of record by googling the name. I talk to any owner personally and ask lots of questions. As with anything you buy, the seller wants to show their product in the best light and the buyer needs all the information that a large purchase warrants. I have rented successfully from Home Away , VRBO, and Airbnb.
gungadin is offline  
Old Jul 1, 16, 11:26 am
  #9  
Original Poster
 
Join Date: Sep 2006
Location: Jersey City, NJ
Posts: 43
The host reviewed me first with a nonsensical non-review... "Thank you Islans enjoy trip back." I don't know what she means by "Islans," but her spelling and grammar are not good. This was visible to me before I wrote my review. I was emailed the review or a link to it and then prompted to write a review. Is that the way it works, the host reviews first?

Thanks for the very helpful comments in evaluating rentals. I'll try Airbnb again now that I'm a little more knowledgeable.
gsusser is offline  
Old Jul 1, 16, 11:53 am
  #10  
 
Join Date: Mar 2007
Location: SJC/SFO
Programs: UA 1MM/*A Gold, WN A+ CP, Mar LT Tit, IHG Plat, HH Gold
Posts: 5,292
I share OP's skepticism of reviews on AirBnB given how predominant 5 star raves are. I can think of many plausible hypotheses for why this happens. Regardless of the reason(s) they got that way, the fact is that inflated reviews are close to worthless.

When browsing a property on AirBnB I look very closely at the owner's pictures, street maps, and Google Street View. Usually I also email the own to confirm details about the property, such as whether there's private access from the outside and whether all the "bedrooms" are fully enclosed by solid walls with proper doors! Probably about one-third of the time the owner never answers. Another one-third of the time the owner becomes hostile after even one such question. Ah, well. At least I'm weeding out the problem landlords in advance!
darthbimmer is offline  
Old Jul 1, 16, 12:52 pm
  #11  
 
Join Date: Jan 2009
Location: London & Sonoma CA
Programs: UA 1K MM, Hertz PC, BAEC Silver
Posts: 8,003
Originally Posted by milepig View Post
Airbnb reviews are only published after both the host and guest have reviewed each other. This probably inflates the rating.

If I had a bad experience it's likely the host knows it and by not reviewing me they avoid my review going public.
This is not the case. The guest review will be posted after fourteen days, come what may. The fourteen days is the time period during which reviews can be written and the delayed publication is designed so that a review, of either the owner or the guest, is not influenced by the other.
lhrsfo is offline  
Old Jul 1, 16, 12:57 pm
  #12  
 
Join Date: Jan 2009
Location: London & Sonoma CA
Programs: UA 1K MM, Hertz PC, BAEC Silver
Posts: 8,003
Originally Posted by darthbimmer View Post
I share OP's skepticism of reviews on AirBnB given how predominant 5 star raves are. I can think of many plausible hypotheses for why this happens. Regardless of the reason(s) they got that way, the fact is that inflated reviews are close to worthless.

When browsing a property on AirBnB I look very closely at the owner's pictures, street maps, and Google Street View. Usually I also email the own to confirm details about the property, such as whether there's private access from the outside and whether all the "bedrooms" are fully enclosed by solid walls with proper doors! Probably about one-third of the time the owner never answers. Another one-third of the time the owner becomes hostile after even one such question. Ah, well. At least I'm weeding out the problem landlords in advance!
I think you need to be sceptical of all online reviews, not just AirBnB. They are written by non-experts who are not trained to distinguish between real and imaginary problems or major and minor issues.

As an owner of a property on VRBO (it's also listed on AirBnB), I am still irritated by the one four star review we have received - the reviewer docked a star because of the difficulty of driving the access road, something which is disclosed prominently both in the listing and in the response email. And yet, on the few occasions when there have been problems (most recently the previous guests broke the dishwasher and we didn't have time to get it fixed before the next guests' arrival) people have been very understanding. Just a little example of why you shouldn't be too influenced by the star rating system.
lhrsfo is offline  
Old Jul 1, 16, 5:15 pm
  #13  
 
Join Date: Apr 2015
Location: YVR
Programs: UA Premier Platinum
Posts: 3,319
I have two different review standards for AirBnB depending on whether it's a "real" AirBnB (i.e. person renting a room in their home or their summer/vacation home) vs. the many "AirBnB hotels" that are professionally managed, often by an agent managing 10+ units and have been purchased for the sole purpose of being rented on AirBnB.

For the former I don't have as high expectations and will basically give 5* to any host who is making a good effort with no major deficiencies or inaccuracies in their listing. For the latter I hold them to the same standard as a hotel and will review harshly if the place obviously hasn't been cleaned since the last guest, if the furniture/bedding/kitchenware is inadequate or isn't clean, if communication is poor and they are not available to answer questions, etc.

For example, if I am renting an extra bedroom from someone in their home, I don't expect their entire home to be in the same state of industrial cleanliness as a hotel. However, if I'm renting what is effectively a hotel suite from a professional property manager, it's absolutely unacceptable to have dirty dishes in the cupboards. Likewise, when dealing with a "real" host, they may be unavailable at certain times of day for example if they are at work. When dealing with a full-time property manager, they had better pick up the phone when I call and if not, return the call immediately, the same as a hotel front desk.
trooper, scottpenderson and Agosti like this.
eigenvector is offline  
Old Jul 1, 16, 7:05 pm
  #14  
 
Join Date: Mar 2007
Location: SJC/SFO
Programs: UA 1MM/*A Gold, WN A+ CP, Mar LT Tit, IHG Plat, HH Gold
Posts: 5,292
Originally Posted by lhrsfo View Post
I think you need to be sceptical of all online reviews, not just AirBnB. They are written by non-experts who are not trained to distinguish between real and imaginary problems or major and minor issues.
Indeed I do exercise an appropriate degree of skepticism in all things. Just what degree is appropriate for a ratings system varies from site to site. You may read my previous post as stating that I read AirBnB's ratings with more skepticism than other review sites because it is so skewed toward the positive.
darthbimmer is offline  
Old Jul 2, 16, 2:30 am
  #15  
 
Join Date: Mar 2014
Location: MUC
Programs: Compulsory Traveller
Posts: 273
Originally Posted by eigenvector View Post
I have two different review standards for AirBnB depending on whether it's a "real" AirBnB (i.e. person renting a room in their home or their summer/vacation home) vs. the many "AirBnB hotels" that are professionally managed, often by an agent managing 10+ units and have been purchased for the sole purpose of being rented on AirBnB.

For the former I don't have as high expectations and will basically give 5* to any host who is making a good effort with no major deficiencies or inaccuracies in their listing. For the latter I hold them to the same standard as a hotel and will review harshly if the place obviously hasn't been cleaned since the last guest, if the furniture/bedding/kitchenware is inadequate or isn't clean, if communication is poor and they are not available to answer questions, etc.

For example, if I am renting an extra bedroom from someone in their home, I don't expect their entire home to be in the same state of industrial cleanliness as a hotel. However, if I'm renting what is effectively a hotel suite from a professional property manager, it's absolutely unacceptable to have dirty dishes in the cupboards. Likewise, when dealing with a "real" host, they may be unavailable at certain times of day for example if they are at work. When dealing with a full-time property manager, they had better pick up the phone when I call and if not, return the call immediately, the same as a hotel front desk.
Exactly. One of the best-thought-out posts I've ever read on FT.
ritesa is offline  

Thread Tools
Search this Thread