Go Back  FlyerTalk Forums > Travel&Dining > Travel Tools
Reload this Page >

Screenscraping Airfares

Community
Wiki Posts
Search

Screenscraping Airfares

Thread Tools
 
Search this Thread
 
Old Mar 14, 2015, 12:39 pm
  #1  
Original Poster
 
Join Date: Apr 2014
Location: AMS
Posts: 10
Screenscraping Airfares

Hi,

for the CS guys here on the forum, what is in your experience the best source for airfares?

I'm either looking at python's Scrapy or Casper with javascript. I have some trouble getting through the OTA's security.

Thanks ^
MTTH is offline  
Old Mar 14, 2015, 1:03 pm
  #2  
 
Join Date: Jun 2008
Location: BER
Programs: BAEC GGL/GfL, Lufthansa SEN, Hilton Diamond, misc other stuff
Posts: 1,374
Check what's actually legal in your jurisdiction.
My personal favorite is to look at (not to say reverse engineer ;-)) iPhone/iPad/Android apps. The REST APIs are often an easy source and make it much easier. Fire off a couple of requests and get a JSON file in return.
http://json.parser.online.fr/ is also a nice tool to throw a JSON at for a first look.
El_Duderito is offline  
Old Mar 21, 2015, 10:23 am
  #3  
 
Join Date: Mar 2007
Posts: 618
With all the SPA/Ajax stuff these days, I find it easier to just use PhantomJS and parse the resulting HTML instead of reconstructing raw HTTP requests.
blahter is offline  
Old Mar 21, 2015, 10:28 am
  #4  
 
Join Date: Jul 2014
Location: Loud, dark, warm, lots of ethernet cables, and in some rack space.
Programs: AA:EXP
Posts: 369
Originally Posted by El_Duderito
Check what's actually legal in your jurisdiction.
My personal favorite is to look at (not to say reverse engineer ;-)) iPhone/iPad/Android apps. The REST APIs are often an easy source and make it much easier. Fire off a couple of requests and get a JSON file in return.
http://json.parser.online.fr/ is also a nice tool to throw a JSON at for a first look.
Most legal gluck usually say it has to be used for personal activities. Not going to open the door, but yeah that's one of the tricks.
Server is offline  


Contact Us - Manage Preferences - Archive - Advertising - Cookie Policy - Privacy Statement - Terms of Service -

This site is owned, operated, and maintained by MH Sub I, LLC dba Internet Brands. Copyright © 2024 MH Sub I, LLC dba Internet Brands. All rights reserved. Designated trademarks are the property of their respective owners.