FlyerTalk Forums

FlyerTalk Forums (https://www.flyertalk.com/forum/index.php)
-   Delta Air Lines | SkyMiles (https://www.flyertalk.com/forum/delta-air-lines-skymiles-665/)
-   -   Delta computers down (https://www.flyertalk.com/forum/delta-air-lines-skymiles/1783354-delta-computers-down.html)

nevansm Aug 8, 2016 5:16 pm


Originally Posted by teCh0010 (Post 27035007)
Having multiple copies of the data replicated to multiple data centers isn't the problem. It's very easy to maintain the integrity of the data with replication, the problem is leveraging the data to make the system available.

If data center A burns to the ground it's an easy call to declare an disaster and begin recovering at data center B. If data center A is still there, there is just an issue with the ATS / switch gear, then it becomes a tougher decision. How long will it take to change IPs, DNS, start the application up at the DR site, confirm data consistency, test the application and make it available? Is that longer that it will take to just get the broken DC running?

Applications that just keep running in the event of a data center going offline are normally what would be considered cloud native, they create availability at the application layer not at the infrastructure layer. That works great for many types of business, but the types of transactions done by airlines and banking don't work well with that type of application architecture. So you need to build large monolithic applications due to the nature and volume of the business transaction. These applications can't create availability at the application layer, it must be provided by a very redundant infrastructure layer.

Agreed that there are some legacy issues with a PSS as whole, but that's only by their own doing. Not logically moving towards nTier, too much reliance on legacy platforms, and an unwillingness to better architect a solution.

Let's get this straight... cloud services like FB, Twitter, and instagram are way more transactional than airlines. Yet their outages are more rare and less impactful because they've architected or re-architected apps better ways.

Airlines (and other large orgs to some extent) have resisted this change because they don't think it's worth the investment in moving away from their legacy systems. It won't happen overnight, but it truly has to eventually. SABRE and travelport/worldpan already offer cloud versions of their apps that have a lot of these benefits. So it's not like it's impossible.

It's simply an issue of spending money.

kenn0223 Aug 8, 2016 5:32 pm


Originally Posted by nevansm (Post 27035218)
It's simply an issue of spending money.

It's likely more about the devil you know vs. the devil you don't. The backend software that banks, airlines, utilities (my industry) use have been vetted for decades (some for close to 50 yrs). The biggest issue with flipping to a new system is not the cost of the upgrade but the unknowns. It many cases it's lower risk to improve the existing system than start at v1.0.

Related to the incident. I agree with others that the true cause will take days. However, if GA Power's statement is accurate a switchgear failure does make sense and, in many environments, the switchgear is a hard to avoid single point of failure. Unless you have redundant wires going to the systems there has to be a point were you switch between primary and redundant systems. This is often done at the switchgear where you are switching the load between the grid and a back-up system(s). It's a fairly simple piece of equipment but the recovery plan is often having a spare to swap in. Based on the fact this was solved in hours and not weeks it could very well be that is exactly what they did. But, it could also be something entirely different.

Finally, having lived as a United flyer (GS) during the heat of their Intergration (that included several computer meltdowns) I am VERY impressed with how Delta has managed this. The only thing I got from UA during these sorts of incidents was a health serving of attitude (search UA's forum for "meltdown" for a sampling). They didn't give out food, didn't provide any real public statements, no CEO same-day video, no vouchers, nothing. My sense here is DL is really trying hard to get things back on track and keep people reasonably happy.

umich07 Aug 8, 2016 5:39 pm

As an electrical engineer that works in critical facilities, I have to say that this is a big failure. A switchgear failure is not an excuse. There should be auto-transfer switches, UPS units, emergency generators, etc. that keep critical equipment online.

My guess - poor maintenance of electrical systems. Which is kinda surprising for someone as regulated as an airline, but then again, this is back-end stuff and not planes in the air. If your systems aren't tested, when something actually happens, you are not going to get the result you intended. My guess - primary power failure was followed by failure of a transfer switch, or batteries, or generator, etc.

bennos Aug 8, 2016 5:43 pm


Originally Posted by nevansm (Post 27035218)
Let's get this straight... cloud services like FB, Twitter, and instagram are way more transactional than airlines. Yet their outages are more rare and less impactful because they've architected or re-architected apps better ways.

Airlines (and other large orgs to some extent) have resisted this change because they don't think it's worth the investment in moving away from their legacy systems. It won't happen overnight, but it truly has to eventually. SABRE and travelport/worldpan already offer cloud versions of their apps that have a lot of these benefits. So it's not like it's impossible.

It's simply an issue of spending money.

Well maybe, but of course it's not that simple. The airlines have legacy systems, business logic, etc dating back something like 50 years at this point. Facebook is 12 years old, Twitter is 10, and Instagram is 6. That alone makes everything much, much more complicated for the airlines.

The airlines actually sell products in a meaningful fashion. There are compliance obligations and contractual obligations (meaning tickets sold to passengers) that must be carefully maintained. That makes things much more complicated than the other examples, where generally speaking you are the product.

And frankly, if fb, twitter, or instagram go down it's not nearly as big of a deal, meaning those companies can generally be less risk averse. When an airline system goes down, people get stranded, cargo gets delayed, unaccompanied minors need to be taken care of, equipment gets out of position, and so on. The other companies generally just need to get the servers operational again and issue an apology.

I'm obviously simplifying things a bit, and I would agree that the airlines have probably underinvested/moved too slowly to update, but social media companies have very little in common with airlines when it comes to technical requirements.

ajg276 Aug 8, 2016 5:49 pm

I've been at LAX most of the day. Incredibly poor service, borderline rude. Not a good show. My flight didn't show on monitors and was not updated on app, had to ask a diamond representative to call operations after I was told (twice) it departed even though I was monitoring the gate from the crown room and knew that was not the case.

DiverDave Aug 8, 2016 5:53 pm

I wonder when they will start posting delays to tomorrow's flights. I expect long after I am gone to bed.... :(

David

homanga Aug 8, 2016 5:54 pm

Any word on compensation yet my 1877 ended up cancelled thankfully on SW out of MDW now not bothering to call yet for reimbursement/voucher but curious experience so far?

Stgermainparis Aug 8, 2016 6:01 pm

Question regarding a domestic DL delay after EU departure. We flew LHR-ATL this morning on VS (were ticketed via DL though). That flight was not delayed. Our ATL-MEM has been delayed and rebooked and delayed and rebooked again. We were scheduled to arrive at final destination at 6:24. We now will arrive at 9:40, assuming of course that the flight still takes off (we are on a new flight #). Are we eligible for (EC) 261/2004 compensation for 3+ hour delay?

In short: EU-USA flight no delay, USA-USA flight delay of 3+ hours to final destination.

audidudi Aug 8, 2016 6:12 pm

How would flying on DL have only earned 190 miles when any DL segment is 500 miles minimum.

bubbashow Aug 8, 2016 6:21 pm

...good luck with those distance-based miles. How are they with trans-oceanic business cabin redemption? Oh...wait. OK...how are they with Premium Economy redemption to Europe? Oh...yeah. What about coach to Hawaii? Hmmmm...some problems there. Basic economy to Australia....that's right their A320s can't take you there either.

I guess if your idea of award redemption is Cancun or the DR, should serve you well...so long as you will fly their version of coach.

KDCAflyer Aug 8, 2016 6:22 pm


Originally Posted by Stgermainparis (Post 27035355)
Question regarding a domestic DL delay after EU departure. We flew LHR-ATL this morning on VS (were ticketed via DL though). That flight was not delayed. Our ATL-MEM has been delayed and rebooked and delayed and rebooked again. We were scheduled to arrive at final destination at 6:24. We now will arrive at 9:40, assuming of course that the flight still takes off (we are on a new flight #). Are we eligible for (EC) 261/2004 compensation for 3+ hour delay?

In short: EU-USA flight no delay, USA-USA flight delay of 3+ hours to final destination.

No, because the delayed flight did not depart from the EU.

bubbashow Aug 8, 2016 6:22 pm


Originally Posted by homanga (Post 27035335)
Any word on compensation yet my 1877 ended up cancelled thankfully on SW out of MDW now not bothering to call yet for reimbursement/voucher but curious experience so far?

Yes....$200 voucher for anyone cancelled today or a greater-than 3 hour delay.

nwflyboy Aug 8, 2016 6:27 pm


Originally Posted by RustyC (Post 27034479)
...Thanks to these MERGERS and having just three major airlines, the pax get treated worse, the FF programs have been greatly devalued for the vast majority of pax, and if any one goes down then the rest of the system might not be able to help out that much - or at least there's not a good plan.

Too Big To Fly.

MasterGeek Aug 8, 2016 6:28 pm

Which Delta flight departing soon from EU can I book now and will be likely cancelled or delayed more than 3 hours ?

Stgermainparis Aug 8, 2016 6:32 pm


Originally Posted by WWads (Post 27035414)
No, because the delayed flight did not depart from the EU.

Lots of info on Internet that states differently. Anyone else?


All times are GMT -6. The time now is 5:39 pm.


This site is owned, operated, and maintained by MH Sub I, LLC dba Internet Brands. Copyright © 2026 MH Sub I, LLC dba Internet Brands. All rights reserved. Designated trademarks are the property of their respective owners.