drvannostren

I'm not saying the ends justify the means, but I liken this to the call centre. If you're ALWAYS experiencing higher than normal call volumes, you need to adjust your definition of normal. The difference here, is it doesn't require more human employees to accept more traffic. If you're finding that searches are up 100x hypothetically, wouldn't you want to figure out why, and then deliver the best experience and results possible to your customers? If you could display multiple results at once, I wouldn't have to search each one individually. Or if you put up better safeguards to stop scraping, rather than suing, you could see how much your traffic went down and/or stayed there and then decided what to do going forward.

I can't say I know the IT department, I'm sure they're nice, hardworking people just like anyone else, but you can't just establish a website and then kick your feet up and not adjust, improve, tweak. They've added this ability to cancel/change AP bookings, in some cases, that's a tangible way you can alleviate some traffic from the call centre. But it's taken forever to roll out and is only available to some bookings, presumably all AC metal ones? I don't know, but I know my flight had 1 TK leg cancelled and it was easy to replace, plenty of options available but I was forced to call and waste someone else's time to do it and my own cuz I couldn't just do it overnight online.

Interesting, wouldn't that be different though, because a blog is a creative endeavor, so you'd be stealing creative content. Which is different than data isn't? I'm not even trying to just push back I'm genuinely curious. If I have complex math on a site and you sell an explanation with the math, I wouldn't think of that as stealing because nothing about the data would be proprietary would it? This is just information being aggregated and put on to a new site, that has a paid membership. Maybe I'm totally barking up the wrong tree, but I guess in my mind data and creative content would be different things and one would be an issue and the other not, as long as it's not secret data, which in this case isn't.

Pseudo Nim

How would it affect conversion if, in order to book, you�re required to log in anyway - and presumably, for MOST people searching, they already have miles locked into AC, hence being a captive audience? I would argue if the number of people who say �ugh, logging in to do a simple award search is annoying� is close to 100%, and the number of people who subsequently abandon and go elsewhere on that basis alone is absolutely zero.

Sarahli2

100% correct. I will be honest, this is a topic that is very hard for non technical people to have just because they... have no idea what they are talking about. That AirCanada lawsuit is a bunch of horse crap.

Dear lord. Pretending to be an API? Reversing an API is less load on ACs systems than scraping (and they aren't pretending. The authorization is from the scraping)

canadiancow

There's the legal aspect, the ethical aspect, and the practical aspect.

When I launched acrewardsearcher.cowtool.com, you didn't even need to log in to use it. Then I noticed... issues. But without people logging in, I couldn't even track the issues very easily. So I required you to make a free account and log in. Then it became pretty clear who was a bot and who was real, so I banned the bots. But they just made new accounts. So I restricted access to new accounts (but then that was hurting the ability of people to use other cowtools, so I changed it so you could make an account, just without access to the reward search). And then if I blocked you, you were basically done (though there was a black market for cowtool accounts).

In my case, there was a tangible cost to increased load on cowtool. But it was very aligned with AC's "costs", in that as my traffic went up, my costs (and presumably AC's) went up. But my revenue stayed at 0. So everything I did on that front benefited both entities.

But if I had charged for it... well now I'm incurring costs, but making a profit. But I'd be causing AC to incur costs as well. And now the incentives are misaligned. I'd want more traffic, because it would mean more revenue. But AC would want less traffic, because it means increased costs.

But a lot of people seem to be missing the bigger picture that they've clearly stated numerous times. Let me throw some speculation into it (i.e. I don't know this for sure, but it's plausible based on what they said).

The typical Aeroplan customer is a FOTSG who lives in Canada. Presumably most of their family are in Canada. Their searches are YYJ-YHZ or similar. Those searches can be done within AC's system. There are costs for those searches, but they're probably very low.

The typical seats.aero/cowtool/etc. user is a "FlyerTalker". We want to go somewhere exotic on a premium airline. If I search SFO-BKK, it's going to require querying basically every one of Aeroplan's partners. AC has no control over TG's backend. AC cannot make it bigger/better/faster.

Maybe AC has to pay Amadeus for those searches. Maybe they have to pay TG. Maybe it's not "dollars", but some other agreement that "if you have 8 million members, you can only hit our system 50,000 times per day".

And here we have thousands of people searching southeast Asia trips with one click, all hitting TG's backend.

AC might be able to handle it. They might even be okay with it. But if TG comes to them and says "you're overloading our system, and it needs to stop or we're cutting you off"... then what. AC did very publicly state that when they tried to launch a calendar[1] they received calls like that from at least one partner.

So even if it's completely legal, would you rather have seats.aero/cowtool, but only half the partner airlines? Or would you rather not have those services, but retain all the partners (with the full availability we were seeing a year ago)? It's not as simple as "AC's backend can't handle it".

That all being said, I don't really like the tactic AC took. Ian said (in this thread) "We attempted to work with Air Canada several times, including offering to change how our scraping worked, but they refused to work with us..."

I tried to talk to them about cowtool for years. I had (well, still have, but it's irrelevant now) a list of questions I wanted to ask them. The very first was "Are you generally okay with it existing?" If they weren't a fan of these things, they could have sent an email. I can't speak for what Ian would have done, but I was willing to acquiesce to their requests, had I ever received any.

They went (over the course of years) from comments like "just because some dude can scrape our site" to "unauthorized systems illegally accessing our site" to (within a week of that comment) sending out a cease and desist. I suspect if they'd just sent Ian an email like "Hey, seats.aero is causing us some issues, can we talk to you about it?" there at least could have been a conversation to see what could be changed.

But AC seems to always take the nuclear option. I didn't know exactly what was coming, but the "unauthorized systems illegally accessing our site" comment certainly led me to believe it was approaching that point.

Also, I still doubt seats.aero is even 10% of their bot problem. I suspect it's substantially less than that. But I'm not convinced they have the ability to determine where the requests are coming from.

[1] AC has also said they have tried very hard to eliminate phantom availability, and when I use other airline's calendars, I find it's very common to click a "70k points" date, and find nothing under 150k. Or vice versa. So I can't speak for how other airlines do it, or if AC could have done a "crappy calendar" that wouldn't have had these problems with their partners.

irishguy28

Only if seats.aero is a mindreader and knows in advance the precise subset of routes and dates its users will want to search for, and ensures it has already searched and cached only those routes that will be "needed" that day.

If seats.aero has instead searched for other city pairs and dates that nobody bothered to check for since they last searched for those city pairs and dates and scraped that data into their cache - it hasn't limited the number of baseline searches; it has made unnecessary searches.

(If seats.aero was a mindreader or could somehow foretell future customer demands, I'm sure it would be dedicated to a far more profitable and lucrative endeavour than scraping award booking sites!!!)

Sarahli2

Anyone know what point.me and roame are up to with this? I remember reading a bit ago that pointme had gotten legitimate API access. However, considering their load times I find that to be highly suspect.

RatherBeInYOW

It is exactly technically correct. Open an incognito browser and do a search for award visibility on ac.com and tell me what you see. This information is made available publicly by Air Canada on the open internet. This is black and white.

If you put data on the internet without requiring authentication then people can come and get it. If Air Canada wants to restrict this information then they can put it behind authentication. There is not some magic solution here, nor have I pretended there is one. This website is not "knowingly and fraudulently present identification to misrepresent who you are, such as stealing someone else's credentials and logging in with them" - it is hitting a public endpoint that AC makes available and collecting the resulting data.

The context is being explicitly misrepresented. Using Akamai Bot Manager is not "preventing unauthorized access".

Air Canada doesn't get to put information on the public Internet and then say "you can't access it" - this has been pretty effectively settled.

I guess the bottom line for me is that the public internet only works because it is interconnected, and this needs to be protected. There are ways to make information private, and Air Canada chooses not to use them here. Trying to use terms of service, lawsuits, etc. to restrict access to information you make public has been pretty effectively upheld as wrong by the courts in the US. The EFFs amicus brief on this from the last court case reflects my feelings pretty well, and I'm reasonably confident that on at least these claims Air Canada will lose, although of course IANAL.

I'll bow out of this now, ciao.

GrayAnderson

I'm not a lawyer and I'm only going to express a layman's view, but it would seem that between including a disclaimer on the front page and the logo use being, I believe, limited to a little logo next to the airline name is probably not likely to cause much, if any, confusion. "We are not affiliated with any airline, and besides we list a bunch of direct competitors' space alongside one another" seems to be a decent rejoinder. "Golly gee, AC is showing me how to find award space on Delta! How nice of them!" seems absurd on its face, and the fact that they direct you to the airline website to do the booking doesn't help here, either. Trying to allege that the site misrepresents AC's stuff also seems strange since it's a direct scrape. If they're only taking data from AC's website, not altering it, and posting it...then wouldn't the only way they would be misrepresenting what AC is offering be for AC to be misrepresenting it?

Basically, the Lanham Act stuff feels like it's not going anywhere beyond an injunction to stop using AC's logo (which I'd note, at first glance, seems to have just been a dumb thing for the website to do...that's just begging for a nasty twist with the airlines) since claiming a demonstrable harm there seems quite tough.

The other stuff is harder to predict in my mind, but as others have said...showing actual damage is likely to be tricky, especially if they can credibly claim that they're saving users from making lots of inquiries. I also have to wonder, if AC knew that seats.aero was the source of a lot of inquires, why they didn't send them a cease-and-desist letter back in May rather than (apparently) deciding to wait until they'd exhausted all other avenues to try and make this stop? "We spent a lot of money on trying to make them stop before we asked" doesn't feel like a great argument, and AC themselves indicate that they knew that some chunk of traffic was coming from seats.aero. [That their demand was ultimately rejected probably helps their case, but it would seem that if nothing else, AC should have moved in that direction sooner, even if in parallel with pursuing technical solutions.]

[I know it is a legal technicality, but I love how AC nominally restates a bunch of claims like eight times even if some of the points are irrelevant to a given complaint.]

Sean Peever

Have we worked together on a case before? You sound exactly like someone I have.

It's evident that there's a significant disconnect in our understanding of the nuances around APIs and data access. Allow me to clarify some critical points:

Firstly, let's address the misconception that API data is equivalent to publicly available information. An API, or Application Programming Interface, facilitates structured interactions with an application's database. These transactions occur through designated queries and culminate in data being returned upon a verified request. This is fundamentally different from data freely available on the internet for public consumption.

Secondly, the assertion that anything on the internet is "public" is an oversimplification that overlooks the intricacies of data access and ownership. While an API endpoint may be accessible online, it is in no way an invitation for unbridled public use. Companies establish these interfaces for specific interactions and have the absolute right to regulate access.

Finally, when a company explicitly blocks access to its servers and communicates that certain activities are unauthorized, persisting in those activities crosses clear ethical, moral, and legal boundaries. Actively circumventing these blocks, especially while publicly acknowledging them, leaves no room for ambiguity: it's unauthorized access.

It's concerning that these points have been either mischaracterized or misunderstood in the discussion thus far. Whether this stems from a lack of understanding or other motives, it's crucial to align the conversation with technical and legal realities.

oliver2002

GDS/CRS allow participants a number of searches/queries and after that the counter is started. Airlines are probably not restricted in querying their own inventory at all, but when they query other airlines inventory, they are (probably) treated just like a major TA/OTA. Such restrictions also has legal reasons, as airlines can't snoop on their competitors inventory/pricing in a large scale systematic way for anti trust implications. (Remember that AC for example only has ATI for TATL traffic with UA & LHG).
Some search engines cache previous searches for some time not to waste queries which causes the 'phantom' inventory problems people have reported. Airlines also tend to protect themselves from barrages of queries from known players (Aeroplan & Lifemiles are on the top of the list of these) by either denying direct/generic queries (only longsell/manual queries allowed) or putting the office IDs behind the web engines of these players on customization lists within the Altea Dynamic Availability module. Sometimes these lists are reset by mistake (example LX F awards available on Aeroplan), sometimes the dynamic availability is tweaked to let go of some award space that would otherwise go to waste (example LH F awards at T-14 or 7d). TG & SQ sometimes allow automated queries via the Amadeus Altea / *A CITP system, but sometimes move to manual/longsell only. UA (Shares) and TK (Sabre) don't use the *A CITP directly as their CRS, but have complex EDI interfaces that communicate between their CRS and partner airlines' CRS or the *A CITP which often breaks down due to configuration changes on either end which don't get fixed with high priority.

Speaking of priority: the sale of tickets on the airline website is priority #1 thru #99. Getting rid of Aeroplan point inventory is probably not the top priority of the people that run the show and hardcore redemption searchers slowing down the revenue sales show are most likely not seen as the core audience. I'm sure some people at UA/AC regret putting the 'search for awards / with points' option on their general sales web page.

YOWgary

Or, as my old boss used to say, "In any disagerement, there's what actually happened, there's how the other guy feels about it, and then there's what he's going to want to DO about it, and as often as not those will be three different things."

Cow's right on the money with this assessment:
I suspect it's as likely as not that there wasn't ever a set agreement about "you can have X many search hits per month", and there may literally never have been a conversation about this until someone at [Other.Airline] called a meeting to ask why their partner search traffic had been multiplying massively month-over-month all year.

AC execs have said publicly, and repeatedly, that both calendar search and third-party tools have led to unhappy phone calls from their partner airlines, and that alone I don't find that terribly hard to believe. I don't find it terribly hard to believe that AC feels they have been done measurable harm through the creation of friction with partners they consider valuable.

I do have an awfully hard time connecting the dots from that to words like "illegal", but I'm not an IT lawyer. I do share the view that it seems like there could have been productive conversations about how to make this kind of tool manageable; AC has certainly shown willingness to partner with third parties in other areas, like delay compensation.

...but many of us have worked in jobs where one exec, or a boardroom full of people, decides that the only course of action is "our way or the highway", and it sounds like in this case, the relevant decision-makers chose not to engage with creators of third-party tools.

As I've said throughout this conversation, I skew towards Occam and Hanlon until direct evidence shows otherwise, and "nobody foresaw this problem, it blew up with the partners and now they're scrambling to patch it up" is a theory I have yet to see contradicted.

esplanades

I don't see the fundamental difference from a PHP/server-side page that displays data directly and an HTML page that uses JavaScript to call an API endpoint to display the same data. Both are publicly accessible; whether that information is being accessed through a browser or `curl` is immaterial to its public-ness.

And they absolutely can, by issuing authorization tokens that only allow access from that user's browser or IP, for a limited amount of time, or requires a logged in session token that the API endpoint can verify, for only particular searches, etc.

scubadu

I certainly am no expert in this area, but I think there are layers of complexities in this that can't just be "hand waved" away because people want/demand access to award availability from seats.aero

First, Avianca Lifemiles does require an account/login to search award availability and yet seats.aero had been providing that data. How do you explain that one away? (hint: mysteriously, without any explanation, Aviana Lifemiles has been conveniently "greyed out" now and no longer provides access on seats.aero) The founder claims that no other airline programs have complained, yet Aviana Lifemiles, who does require an account, is now no longer available. Hmmm... why do you suppose that is? Additionally, it's possible that other programs seats.aero supports require accounts, I'm not sure and I'm too lazy to check all the programs I don't use. So one of your arguments was this data is publicly accessible and not "behind closed doors" but in the case of Avianca Lifemiles, it was not "publicly available"and yet seats.aero was still presenting it. Your assertions cannot simultaneously be true in one case but not the other.

Second, like others, I believe there is a difference between webscraping for say, personal use vs. web scraping for commercial use. Seats.aero is scraping data and then selling users a subscription to obtain that data (at least for awards more than 90 days out). I am certainly no legal expert and I'm humble enough to say I don't know if that is legal or not, but I believe it certainly merits understanding. Courts have often provided legal protections for personal use vs. commercial use. For example, courts allowed home users to record TV shows and movies when VHS came out for personal use. However, I couldn't just record a TV show or movie and make a bunch of copies and sell them to people walking in front of my house. This isn't a perfect analogy, but my point is that courts have certainly recognized rights of a citizen in the area of personal use that do not necessarily extend to commercial use.

I'm just perplexed by the level of certainty projected by people in this thread (irrespective of which side one is on). There is an awfully lot of "no, you are absolutely wrong and I am absolutely right" type of posts. Unfortunately, this is just not an area that is as black and white as a few posters in this forum are implying and attempting to yell the loudest doesn't make it so.

Finally, though I certainly don't agree that what all large companies do is right, I struggle to believe that Air Canada doesn't have a fundamental understanding of the law here. If things are as cut and dry as all the anonymous internet experts here are claiming, I seriously doubt they would be going down this path.

At the end of the day, if you are betting on the outcome here, you have to ask yourself, are you putting your money on this guy:

https://www.linkedin.com/in/ian-carroll-a56b8758/

or on Air Canada and their legal team?

Regards

Jumper Jack

Well, at least in my limited experience working in product management, the metrics I have seen is any additional step will increase the drop-off
Your assumption is that people who came to AC website already want to purchase a ticket, I dont agree with that belief. My take is that there are quite bit of browsers who are just casually checking out the ticket price, or that they may simply be shopping around. Introducing a login process before the payment page is counter intuitive and remove a % from your top of funnel that might have went through with a purchase.

flyingcrooked

I have no view on the larger dispute, due to ignorance, but on the point quoted, don't you think many large companies with deep pockets undertake lawsuits even if they know the law is against them when they know they can crush/outspend/intimidate the other party? I'm not saying that is what is motivating AC, of course, just that for all I know we know it could be. It's not like if a large company knows the law is not on their side, they automatically throw up their hands and concede.

On the specific topic of AC, it's pretty clear if you look at their responses to compensation claims that they don't care what the law says, they care about what they can get away with, at least on some matters.