FlyerTalk Forums

FlyerTalk Forums (https://www.flyertalk.com/forum/index.php)
-   Delta Air Lines | SkyMiles (https://www.flyertalk.com/forum/delta-air-lines-skymiles-665/)
-   -   Delta computers down (https://www.flyertalk.com/forum/delta-air-lines-skymiles/1783354-delta-computers-down.html)

WIRunner Aug 8, 2016 5:51 am


Originally Posted by dzflyer (Post 27031699)
Today is putting the hurt on the cancellation and on time preformance. Delta the most on time airline if our computers work. No back up generator is the best punchline ever.

I doubt that is the case. It not turning on in time is more likely.

anon2k2 Aug 8, 2016 5:51 am

On Friday I changed my outbound from today to Tuesday. If things are bad enough, I guess that might not be long enough.

PAX_fips Aug 8, 2016 5:53 am

I can take this as a poster-kid for bad planning (and "news") to next customer already - where'll talk about backup power tests, DR offsite locations and so on.

Hilarious handling.

"Airlines in the cloud"? :-)

MSPSKI Aug 8, 2016 5:53 am

Just announced in club. Msp-yyz now boarding.
Only flight at msp that is boarding.

raehl311 Aug 8, 2016 5:55 am


Originally Posted by dzflyer (Post 27031699)
Today is putting the hurt on the cancellation and on time preformance. Delta the most on time airline if our computers work. No back up generator is the best punchline ever.

They almost certainly have two or more. I would bet some component in the failover process failed, and either the generators didn't fire up (whatever mechanism was supposed to turn them on did not) or they did fire up but the power couldn't get to the system.

mother- Aug 8, 2016 5:55 am


Originally Posted by raehl311 (Post 27031658)
For a system of that size, battery power is probably on the order of minutes. So if generators no-go....

More likely some of the power equipment like a panel or switch between the generators and the system failed.

I'm pretty sure that is "something bad... at their datacenter". Mainframes take a while to do a cold start/IPL...

My money want's to be on someone hitting the wrong button trying to leave the raised floor.


Originally Posted by nevansm (Post 27031686)
This I find very hard the believe, unless their multiple ISP's lost power too. There's battery backups for the first 10-12 min, generators to provide endless power, not to mention DR sites. Really don't get this issue. (FYI, IT architect here. If this was my people, heads would be rolling)

If the source of all truth in the universe is your mainframe, running ancient code, they may not have bothered with active/active DR sites, and it may in fact take 3 hours to either get a warm site up and running or get the mainframe in the main DC back up... That is my logic at least.


Edit: Turns out it was 5 hours...

raehl311 Aug 8, 2016 5:56 am


Originally Posted by anon2k2 (Post 27031710)
On Friday I changed my outbound from today to Tuesday. If things are bad enough, I guess that might not be long enough.

It might be perfect if you're willing to take a voucher.

Rosaus Aug 8, 2016 5:56 am

Pretty sure Deltamatic is still on IBM mainframes. They are extremely tolerant machines that do not break easily. It has to be either external systems (networks routers/switches ... or firewall changes) or once in a lifetime operator errors that breaks the actual mainframe.
Meanwhile, being on mainframe makes it much harder to go 'Cloudy' :)

ND76 Aug 8, 2016 5:57 am

The captain of our 0525 CDT flight OMA-ATL just said that Delta's systems are back up and operational, but that landings at ATL are going to be "staggered". He guessed that we won't be leaving here for another 3 hours (approximately 10 am CDT).

They are getting ready to board the OMA-MSP flight right now.

PAX_fips Aug 8, 2016 5:57 am


Originally Posted by raehl311 (Post 27031729)
They almost certainly have two or more. I would bet some component in the failover process failed, and either the generators didn't fire up (whatever mechanism was supposed to turn them on did not) or they did fire up but the power couldn't get to the system.

Careful with words here.. earlier this year a technician (almost) went up in flames ("fired up") when he short-circuited somewhere between mains and backup power (brought down one of two IT-"lines", too - double backup for more winning).

RTW1 Aug 8, 2016 5:58 am


Originally Posted by nevansm (Post 27031686)
(FYI, IT architect here. If this was my people, heads would be rolling)

What a lousy boss you would be, drawing conclusions without even knowing all the facts :td:.

raehl311 Aug 8, 2016 6:01 am


Originally Posted by PAX_fips (Post 27031740)
Careful with words here.. earlier this year a technician (almost) went up in flames ("fired up") when he short-circuited somewhere between mains and backup power (brought down one of two IT-"lines", too - double backup for more winning).

I actually have a buddy who maintains backup power supply systems for a living. His stories of what they'll do for certain can't-go-down critical systems make me queasy just hearing about them.

They apparently work with live components on a not infrequent basis. Touch the wrong thing and yer dead.

SFO777 Aug 8, 2016 6:03 am

Wouldn't be surprised if Bastian blames the ME3 and especially QR for the outage. ;)

Orange County Commuter Aug 8, 2016 6:05 am

I did wake up to a major storm last night at about the time Delta seems to have "died" so that may have been the first "source" of the power outage. Does appear they have back up problems. (But I had a hospital have the same issue a few years ago in spite of "testing" so... it happens!)

scongro Aug 8, 2016 6:05 am


Originally Posted by RTW1 (Post 27031742)
What a lousy boss you would be, drawing conclusions without even knowing all the facts :td:.

Not to go down this route, but I'm in IT as well and I feel the exact same way as the poster that said, "heads would roll". A company like Delta employs quite a lot of people to ensure this exact thing doesn't happen. Millions of dollars in equipment is spent in batteries and generators (and we're not talking about a couple Car Batteries here or Food Truck generators). If they don't work then why was the money spent?

We call these sort of incidents, "resume generating events". Unless this is terrorism or a failure of sorts that will put the entire IT world in awe because of the fact that everything everyone knows about redundancy is wrong, many someones didn't do their job.


All times are GMT -6. The time now is 8:10 am.


This site is owned, operated, and maintained by MH Sub I, LLC dba Internet Brands. Copyright © 2026 MH Sub I, LLC dba Internet Brands. All rights reserved. Designated trademarks are the property of their respective owners.