Go Back  FlyerTalk Forums > Travel&Dining > Travel Tools
Reload this Page >

ITA software scrape

ITA software scrape

Reply

Old Dec 4, 18, 8:57 am
  #1  
Original Poster
 
Join Date: Sep 2004
Programs: LH HON, AF Plat, BA Gold, SPG Plat, HHo Gold
Posts: 138
ITA software scrape

Just curious, whether anybody has succeeded in creating a scraping script for ITA software. I have seen a post over on headforpoints, which suggested it was possible. Any insight?

Thanks,

Jan
Jan@BRU is offline  
Reply With Quote
Old Dec 5, 18, 12:28 am
  #2  
 
Join Date: Jun 2015
Location: SFO/SJC, BWI
Programs: :rolleyes:, DL DM, Mlife Plat, TR 7*, SPG/MR Plat, UA 1K, WN A-List
Posts: 8,533
Originally Posted by [email protected] View Post
Just curious, whether anybody has succeeded in creating a scraping script for ITA software. I have seen a post over on headforpoints, which suggested it was possible. Any insight?

Thanks,

Jan
What exactly do you mean by scraping?

Are you looking for functionality provided by either the Userscript or bookwithmatrix?

ITA-Matrix-PowerTools - Userscript for Orbitz/DL/UA/AA/BA/CZ/IB/LA/LH/LX/TK

BookWithMatrix.com: a tool to easily book with ITA Matrix
Zorak is online now  
Reply With Quote
Old Dec 7, 18, 9:22 am
  #3  
Original Poster
 
Join Date: Sep 2004
Programs: LH HON, AF Plat, BA Gold, SPG Plat, HHo Gold
Posts: 138
Zorak,
No -- that's what I mean. From what I understand, the tools you mention assist you in purchasing specific search results, which indeed is a big help.

Scraping refers to automating requests sent to a website and harvesting the results.
Take for example the recently introduced limitation on ITA to limit the jointly queried departure points to cities in a single country. If you want to do a regional query of flights from all departure points in neighboring countries, you need to several requests.
Scraping would run a script, send out a request, wait for the results to display and than store the results before moving on the next request automatically etc.
It's obvious that google discourages that kind of data harvesting and has put in place a few barriers to do it.... I was wondering whether somebody has managed to successfully scrape ITA nonetheless.

Thanks

Jan
Jan@BRU is offline  
Reply With Quote
Old Dec 7, 18, 11:10 am
  #4  
 
Join Date: Jun 2015
Location: SFO/SJC, BWI
Programs: :rolleyes:, DL DM, Mlife Plat, TR 7*, SPG/MR Plat, UA 1K, WN A-List
Posts: 8,533
I know what scraping means in general, but wanted to understand your specific use case and whether it had to do with booking or searching.

IMO if someone wanted to automate the searching process or aggregate the results of multiple searches, a more direct approach might be to use Developer Tools to try and figure out the underlying search API and then issue customized requests directly to that instead of going via the UI.
Zorak is online now  
Reply With Quote
Old Dec 7, 18, 1:40 pm
  #5  
 
Join Date: Feb 2015
Posts: 175
Same problem discussed in a slightly different way. Here in Western Europe several international airports may be within 150 miles of each other but in four different countries so while the old ITA-Matrix allowed a quick search that has now become more tedious. Any suggestions how a searcher might extend to search 'airports within x miles'
Just a thought

Last edited by gbs1112; Dec 7, 18 at 1:41 pm Reason: spelling
gbs1112 is offline  
Reply With Quote
Old Dec 12, 18, 5:48 am
  #6  
Original Poster
 
Join Date: Sep 2004
Programs: LH HON, AF Plat, BA Gold, SPG Plat, HHo Gold
Posts: 138
Originally Posted by Zorak View Post
I know what scraping means in general, but wanted to understand your specific use case and whether it had to do with booking or searching.

IMO if someone wanted to automate the searching process or aggregate the results of multiple searches, a more direct approach might be to use Developer Tools to try and figure out the underlying search API and then issue customized requests directly to that instead of going via the UI.
Sorry, yes it's about the searching, not the booking.

I appreciate thata developer API might be better, it just so happens that itasodtware closed down its already limited QPX API last month.

So any thoughts on a js or vbnet based routine that jumps the itasoftware anti scraping hurdles would be welcome.
Jan@BRU is offline  
Reply With Quote
Old Dec 13, 18, 8:44 pm
  #7  
 
Join Date: Jun 2015
Location: SFO/SJC, BWI
Programs: :rolleyes:, DL DM, Mlife Plat, TR 7*, SPG/MR Plat, UA 1K, WN A-List
Posts: 8,533
Originally Posted by [email protected] View Post
Sorry, yes it's about the searching, not the booking.

I appreciate thata developer API might be better, it just so happens that itasodtware closed down its already limited QPX API last month.

So any thoughts on a js or vbnet based routine that jumps the itasoftware anti scraping hurdles would be welcome.
There may not be a public-facing API anymore but there's still whatever API their UI talks to. As I said above, my first thought absent any better suggestions would be to use Developer Tools in a browser, sniff whatever requests it's making and try to reverse engineer what it's doing.
Zorak is online now  
Reply With Quote

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off


Thread Tools
Search this Thread
 
  • Ask a Question
    Get answers from community experts
Question Title:
Description:
Your question will be posted in: