julp

The recent cease and desist letters from UA to Expert Flyer and Wandering Aramean have been a real setback for some United flyers. I relied heavily on EF's Award Alerts (at first) and WATT Star Alliance award alerts (after EF got shut down), and not having them has been a blow to my travel planning.

From my understanding the methods to scrape/search the award space are still available, its just that every time United catches a major player using those channels they shut them down. I propose the development of an open source award alert tool that everyone requiring award alerts can run themselves on a home machine or cheap cloud server. The goal is to create a web application that requires minimal set-up and only basic technical knowledge, no programming skills. Anyone needing an award alert could run it on their home computer with a simple MAMP/LAMP/Python/etc installation.

By distributing the search load on United's servers to a large distributed network of home machines (instead of a handful of fixed IP servers) United's ability/interest to detect and shut down those servers should decrease significantly.

Personally I have access to resources to build said web application, but I have no knowledge of how EF and WATT were searching award inventory. My hope is that the FT community can chime in and collaboratively we can design a tool that everyone can benefit from. As an open source tool we could over time even include other airlines in the award search, as members contribute to the Github code base.

Perhaps such a tool already exists, or I am missing some obvious risks. I would love to open this up for discussion and see what's feasible.

AirMiles2001

Sounds like a good way to get the advanced Fare Class mode removed again.

Starman

sounds like a great idea to me -- why should UA care which browser I use, even if a customized one, to access their data using the login they provided to me, as long as I'm not providing services to others.

AirMiles2001

They are going to care that a single ip is generating so many requests within a short period of time. I just hope they only ban the feature from those that abuse it and not all of us.

mduell

The most obvious problem with this plan is UA doesn't know how WATT worked. They sent a C&D referencing a website he wasn't using. If you publish OSS using the same method, UA can see how WATT works and disrupt it.

The best approach I can think of is for someone anonymous to use the WATT method behind a tor gateway, so the requests to UA come from all over the place and the method remains unpublicized.

Starman

I see your point, but if one imitates a KVS architecture (convenient line by line access to currently open space), it's not clear to me that the total number of requests will be a lot different from querying the same data with a browser; I can click Firefox pretty fast. I agree that if you also want EF functionality to create alerts, that creates a lot more queries from the same IP address. But many of us are so maniacal that we used KVS constantly anyway to watch for newly opened R space. I'd be quite happy to have that functionality back, even though not as good as getting alerts.

dsauch

KVS received the same C&D letter and had to remove UA search functionality from the tool. I had the same argument you did - KVS is just a browser, UA shouldnt care. But they did.
May be because KVS made money on it, I dont know.

I understand the idea of the tool is "for good", however it will take no time at all for "brokers" to spun up, ran hundreds and thousands of instances and then sell the data to less technically inclined.

As others said - UA will almost certainly shut down expert mode then.

zebranz

+1

You don't sword fight with Zorro!

DaviddesJ

I'm as frustrated as anyone that UA is not transparent with upgrade availability, but building a distributed systematic effort to violate their terms of service is going to be ineffective and unproductive.

Mike Jacoubowsky

There also be may language in the MP rules that forbids such methods, and the very sort of person interested in award and upgrade availability has a lot to lose if UA decides to close their account and nullify their earned miles.

sbm12

Wirelessly posted (Mozilla/5.0 (BlackBerry; U; BlackBerry 9930; en-US) AppleWebKit/534.11+ (KHTML, like Gecko) Version/7.1.0.1066 Mobile Safari/534.11+)

I was explicitly told that even if I had no ads and no revenue they would still pursue me. I am fairly confident that the source IP of the requests was not a problem. Spreading that out is actually surprisingly easy even without proxies and gateways.

Building an app to run the queries individually rather than centralized is trivial. But, as noted above, distributing raises any number of potential issues. Just making something open-source doesn't make it legal. Plus, the generic clause of behavior detrimental to the program is ridiculously broad and one I'd be very wary of in planning to piss the company off.

mduell

Anyone using their primary UA MP account to do automated/batch searches is an idiot.

Your dog, real or imagined, can get a UA MP account in seconds for free.

flyerbjorn

Please don't do this. It will be the reason we cannot have nice things.

This won't work. Even if you distribute the searches over smaller range (say hundreds) of IPs, automated queries will stand out to UA.

Xyzzy

Amng other things, the web site's T&C state the following:

halls120

I'm better expert mode is gone when they roll out the new website.