HGF

Hey all,

I've been working on a flight finding program that queries ITA's matrix for flights and alerts you of cheap/low ppm ones. I took a break for a few months due to life events, but I am going to be starting up again

Right now, it only does searches out of ORD to any number of inputted destinations between the specified dates for the specified trip length range. It then prints out the lowest ppm and lowest priced trips per destination I believe.

I was working on making it RESTful since I don't like doing UI work. After that, setting up a DB to cache results would probably be best. Matrix likes to throw errors relating to random failures or exceeding quotas a lot.

If anyone wants to help out, I have it on github here: https://github.com/HGF/flight_finder

gnutello

Please consider merging with this project. You'll see that some of the legwork is done already. There are some more recent updates in the branches. They've already put a little consideration in how to store the results in a (NoSQL) database.

https://github.com/mayanez/flight_scraper

HGF

The goals of that project and my project are different. I am not trying to determine a correlation between seat availability and price fare information. I just want to find great flight deals for either mileage runs or cheap vacations by essentially automating the process that everyone already does on Matrix. There also hasn't been an update in over 6 months and a pull request has been pending for over a month.

I also want to do a lot of the legwork, so I can learn.

mgo72

Hello,

What help do you need?

HGF

I need to implement a DB for caching. I am also in the process of making it restful, so someone can crate a separate UI for it. Supporting multi-destination itineraries would be another good thing as well. Or, if you have your own features in mind, do those as well and it can be merged in.

I should create a TODO list in the readme for what needs to be done to give people some guidance

CAETravlr

I have written something similar. My suggestion is to look for apis if you can. You will have your IP blocked by Matrix if they determine that you are scraping their site. I learned the hard way, but probably because I got ambitious and multithreaded it. Look up the QPX api and see if you can get access to that. I am in the process of refactoring now to use that.

HGF

Using the QPX API costs money per query though.

sokolov

One problem is that ITA Matrix searches are only yay deep. You will never see everything, especially if you are searching for periods longer than single days.

Furthermore, results may differ between the web version and the mobile app.

Caching can be interesting for historical comparison, but for current results, I don't know. You would be caching something that comes out of a cache already. :-)

CAETravlr

After the first 50 queries each day I believe. And even then, it is not very expensive. How many queries are you planning to run? Now granted I haven't refactored my program to get the data this way vs scraping matrix like I did before, but I think it is a small price to pay for a more reliable interface and to stay within their terms of use.

sokolov

Would you have the link to the pricing structure?

angatol

.....

bittihuduga

i get error on mac terminal:
python FlightFinder.py
Traceback (most recent call last):
File "FlightFinder.py", line 5, in <module>
from ITADao import ITADao
File "flight_finder-master/flight_finder/ITADao.py", line 4, in <module>
import requests
ImportError: No module named requests

Debonaire

It looks like you're just missing the requests module? Can you pip install requests?

Debonaire

I think this is it: https://developers.google.com/qpx-express/v1/pricing

How does one get seat availability from ITA? Is the solution just to increase the number of passengers (up to 9)? That would significantly increase the number of queries.

Why not just match it with fare class availability via say, ExpertFlyer?

gnutello

My pull request finally got accepted. While the purpose of the project might be different, I see no harm in forking it -- particularly since it already has a NoSQL database integrated. Basically all you have to do is implement more XMLHttpRequests. The code is pretty easy to work with -- at the minimum, you can reuse a lot of its basic functionality as long as you redesign the database "tables".

When I get the chance, I'll take a look at your code and see if there's a fast way to port over some of the other code. You'll probably want to look into using Tor with Python to allow multithreaded queries. Also, when you make requests from ITA, make a driver that repeats requests until you get past the "server capacity exceeded" error, or 1 minute passes.