lhrsfo

I'm not a lawyer but my definition of stealing is depriving someone of their own property. The problem I have with the statement that EF is "stealing" the data is that UA itself makes that data available to people free of charge, albeit in a not-very-useful form. Clearly UA is content for me to access the data. You suggest that I should pay someone to access it on my behalf - that's precisely what I was doing with EF - I paid them $99 and they condensed the data into bits I could use.

And, incidentally, I used it principally when looking for trips fairly far out so that I could be alerted for R space - not so much for queue-jumping as we all know when the R space opens up at T-23.5 or so.

star_world

Fair enough - the accurate statement would be "EF was operating in clear violation of the Terms of Use for the UA website, and was blocked as a result".

The relevant paragraph, as always:

Kacee

"Stealing" is not the right term, but EF was apparently making commercial use of UA's proprietary data in violation of UA's terms of service. That's almost certainly why EF stopped when UA told them to, because they would have lost if UA had commenced legal action. I don't have any sympathy for EF on these facts. I had assumed EF was properly licensing the data from UA, not illicitly scraping it for free in violation of UA's ToS.

star_world

+1 - I had always assumed the same. That's why I have zero sympathy for EF, nor can I see how this is UA's "fault" or an attempt to increase opacity, as some here keep insisting.

FlyWorld

Several others have rebutted your assertion that EF's behavior constitutes theft or hacking.

Bottom line: EF provided a valuable service that we need.

1) Because R space doesn't clear properly or consistently, we need a way to manage the constant failure to perform, and

2) Because R space became so scarce after the CO takeover, we need better tools to find it

UA knows that we relied upon EF to get around these obstacles and rather than fix the obstacles by improving their web site, fixing the UG clearance algorithm, or providing their own alerts, they just shut down EF and made the entire process more opaque and difficult.

Kacee

It's not just UA, most any business would have sent EF a cease and desist letter on these facts. Taking a company's proprietary data without a license and selling it is just not something you can expect to get a away with. You either get a license or you abide by the ToS.

star_world

Do you disagree that EF was operating in violation of the UA ToS? Nobody is rebutting anything besides the technical term to describe what they're doing, you're just grasping at straws.

Everything you posted above, all the things you consider to be mitigating factors, are irrelevant. It doesn't matter if they were set up to specifically find R inventory to give to poor, starving children - the issue was with the way they were doing it.

pete4212

First off UA obviously doesn't care about screen scrapers. UA still lets many other services, most of which charge a fee, screen scrape from United.com. So it can't be just that. UA Insider could have used this opportunity to say that all unauthorized screen scrapers were not allowed or to be blocked by UA, but he didn't. This means that UA doesn't really care about screen scrapers and targeted just EF for some other reason, possibly because of their alerting capabilities, possibly because they were the most visible. Either way it doesn't matter, by not at least publicly stating that they were going to go after ANY screen scraper that charges for their service they showed that they don't really care about it.

Don't forget, EF was the last service to add UA awards and upgrades, not the first. I know I personally ask EF for years to add UA award/upgrade data before they did.

Second, if UA's problem was only how EF got the data, not if EF has the data, then UA should be able to come to an agreement with EF. However EF has stated that "So far, UAL has made not made any offer, or counter offer, of their own." So your assertion must be false.

Third, UA doesn't have a legal leg to stand on. Not only do we know that UA has no problem with us seeing the data (they give us Expert Mode) but you can't copywrite public information, nor is the information confidential. So there is no legal reason why EF can't posses the information, and no, it was not "stealing". Terms of Service's of websites aren't legally enforceable and we should challenge UA to come up with evidence that what EF was doing actually effected the UA.com website in any significant way. EF has had UA award data for years, so it clearly wasn't abusive or significant enough to notice.

So that combined with the tolerance of other screen scrapers makes it clear that UA has a hidden agenda here and is not being honest with their statements. It sounds like some executive stumbled across what EF had and went into a paranoid fit and yelled at someone to do something without considering the consequences of their actions. Typical.

star_world

Why would UA Insider need to say this? Their ToS already says it in black and white. They can shut down any service or individual that violates this - that's their published policy.

Maybe EF was the biggest - maybe they were generating the most traffic so it got noticed - there are plenty of possible explanations, but that doesn't mean that UA can't shut them down. Not calling out other providers of similar services doesn't invalidate this.

That's because UA never had an easy way to show this information on their website in the past.

Walk me through that 'logic' again? UA is obligated to come back with a counter-offer or the assertion is false? Not getting your point.

They don't need a legal leg to stand on. It's not a public service. They're not obligated to provide it. They can shut off access to anyone for any reason they choose - and they publish ToS to provide guidelines on when they are most likely to do so. If you're operating a business that depends specifically on being in violation of a whole bunch of the terms in there, you should expect to be shut down. What does legality have to do with it?

May not be worth the time, money or effort to go after smaller screen scrapers, particularly those like KVS that are distributed and don't perform scripted searches. It's highly likely that the load generated by EF was several orders of magnitude greater than all the other screen scrapers put together. Again, it doesn't invalidate the decision to shut them down.

Kacee

Much of the key data that EF was repackaging and selling, specifically R, IN, and XN inventory, was only made available by UA in Expert Mode, which means you need to log in to your Mileage Plus account to access it. So EF was either using existing MP accounts to obtain that data, creating dummy accounts, or accessing the data in some way that would normally be described as a "hack."

This is not a situation where EF merely accessed data that UA made available to anyone who visited its website. The data (R, IN, XN) was only made available to people who expressly agreed to abide by ToS. UA could absolutely enforce its ToS on these facts.

1P

It seems perfectly clear to me. UA would like EF to enter a formal agreement (by which I understand "contract") with it to make use of the data, and for whatever reason EF is not willing to enter into such a contract.

Perhaps they would have to pay so much to UA under the terms of such a contract that either their service would no longer be commercially viable or they would have to charge subscribers a lot more than they did before. Or perhaps there is some other reason that we don't know about.

Anyone from EF care to comment?

pete4212

KVS does the same and also profits from it, where is the statement from UA Insider about them?

They already did. In response to emails EF has been responding with "ExpertFlyer is attempting to negotiate an agreement with UAL. So far, UAL has made not made any offer, or counter offer, of their own. However, we continue to discuss the matter with UAL in hopes of reaching a mutually agreeable conclusion."

It seems that UA is the roadblock that is holding up the agreement, not EF as they apparently haven't responded to any offer that EF made (hence the "counter offer" statement) nor proposed one of their own.

Because not doing so is tacit approval of others doing the same. It would seem stupid not to as all they succeeding in doing was moving people from one screen scraping service to another. Like I said before, if they really cared and really wanted to stop all screen scrapers they would disavow all of them thereby removing any safe harbor for those of us that wish to use such services. No blocking or C&D necessary, just create the perception that they are going after everyone and destroy the market.

Expert Mode has been on the UA website for years.

All your other points have already been addressed.

star_world

Are you aware of the fundamental difference between the two? KVS runs on a customer's own computer, and uses the customer's own MP credentials to perform the search. There is no large-scale automated batch searching happening, just interactive search that formats the results in a useful way for the user. UA would have to go after each individual customer of KVS. EF operates completely differently - they generate vast numbers of dummy MP accounts, and run automated queries from central servers in large volumes to provide alerts to users. I think you can see why EF is a more natural target for UA than KVS.

There's an explicit prohibition in the ToS - how could it possibly be a tacit approval? Given what I pointed out above regarding the difference between EF and tools like KVS, are you aware of any other services similar to EF that operate in a similar way and haven't been shut down?

pete4212

UA and their lawyers wouldn't care as KVS is a 3rd party making money from displaying UA's data, period. The fact that KVS asks for a MP# and password (which EF never did) is even worse as then it's a "security issue" as the airlines like to say. If EF asked for a MP# and password they would still get shutdown. Case in point: AwardWallet. They do the same as KVS according to you (only access UA.com on behalf of user with users credentials), but UA sent them a C&D anyway. Also, if KVS was so legit, he wouldn't be hiding from the airlines so thoroughly as he does.

Yes, several that screen scrape from web based services:
AwardNexus.com
AwardTravelr.com
wandr.me

AwardNexus in particular both charges a fee AND does automated searching. So as you can see, EF is not unique in what they were doing.

dsauch

You are wrong about KVS - from IT perspective it is simply a browser. Most browsers are free, this one is not. It browses and presents data from world wide web. There is a huge difference between that and hourly searches for many alerts.
They could have added alerts and Ive asked for it - the answer was, UA will most likely not be thrilled about it. And they were right, unfortunately.

What else do you want UA to ban - how about Last Pass mobile? They charge money for displaying web content in their own browser and make it easy by saving passwords - the nerve!

do you have some agenda to prove by naming and shaming other still live services? I am sure UA knows of them in some way, but why highlight them now? Or the moral voice in you wont be quiet until all violators are punished?