Aug 9, 2016, 1:04 pm
Last edit by: TheLifeOfA_NKCM
As most of us ...well, all of us, know, there was a power failure at ground ops in ATL on Monday (2016-08-08) at 2:39AM which lead to mass-cancellations and delays and rebookings spanning as far as next Sunday (2016-08-21) as of Aug 9. Passengers didn't hesitate to display their dissapointment, Twitter, ACS, and DL phone lines were ambushed with customers. In an effort to keep our fellow flyers informed on the incoming changes and information, this Wiki was formed.
You may all feel free to edit it as your leisure as we continue to watch this develop..
If you are currently at an airport right now, your information on the ongoing events and current situation there is equally as valuable. Thanks for your input.
If you have part of an article in your post, please cite it with a URL to it's
origin.
Airport & Support Situation..
NRT: Foam or Blowup Pad, Crackers, and water given out to customers for sleeping prior to hotel voucher last night
MSP: "Security Lines best since reconfiguration"
---
Note:
Some passengers are reporting long waiting queues on DL phone lines. If possible avoid contacting DL CSR by phone (DM and medallion lines are also expected to have a notably longer wait as well) and instead redirect att'n to Twitter which has been said to be notably quicker than other means of assistance, including Need Help Centers and Gate Agents. Fly Delta and Delta.com should also be permitting rebooking on to other flights at this time. If you are unable to take advantage of any of these methods, express add'l patience
Delta Related Updates:
---
August 11th, 2016 // 11:00AM ET - Flights OPs Return to Normal
Flight operations have returned to normal following Mondays outage, airline officials said at noon today. Delta started the day with roughly 25 mainline flights canceledsome due to bad weather in the Midwestand no additional cancellations are anticipated.
The airline is back on its feet, moving past this weeks disruption and operating normally, said Dave Holtz, S.V.P. - Operations and Customer Center. Running a reliable operation is something we pride ourselves on and frankly, that didnt happen this week. But our teams have worked tirelessly to bring back the dependable airline our customers have known us to be and have come to expect from us.
SLC.jpgWith the impact of the power loss is in the rearview mirror, the airline is carefully monitoring the development of a weather system that may hit the eastern third of the U.S. later today, with the possibility of air traffic control-related delays possible in the Northeast. No additional cancellations as a result of weather are expected, however.
The airline plans to fly more than 610,000 people today. By noon Thursday, nearly 2,200 flights had departed with more than 85 percent departing within 15 minutes of the scheduled departure time.
Even so, customers traveling today should check the status of their flight at delta.com or the Fly Delta App. Delta's travel waiver has been extended through Thursday.
Monday morning an uninterrupted power source switch experienced a small fire which resulted in a massive failure at Deltas Technology Command Center. This caused the power control module to malfunction, sending a surge to a transformer outside of Delta, resulting in the loss of power. The power was stabilized and power was restored quickly. But when this happened, critical systems and network equipment didnt switch over to backups. Around 300 of about 7,000 data center components were discovered to not have been configured appropriately to avail back-up power. In addition to restoring Deltas systems to normal operations, Delta teams this week have been working to ensure reliable redundancies of electrical power as well as network connectivity and applications are in place.
---
August 10th, 2016
Travel waiver impacted dates now extended to include August 11th
https://www.delta.com/content/www/en...em-outage.html
---
4:30PM ET // August 10th, 2016 - Delta has canceled 317 flights today, none in the last hour. Of 3,100 flights that have departed during the day, about 2/3 left within 30 minutes of their scheduled departure time.
Delta's travel waiver has been extended. The airline has extended its offer of compensation to customers significantly affected by lengthy delays or cancellations. Many customers have already been contacted by Delta.
Delta CEO Bastian: 'This isn't who we are'
---
2:30PM ET // August 10th, 2016 - Delta has canceled 311 flights today. About 2,540 flights have departed, 70 percent of those within 30 minutes of the scheduled departure time. Delta operates nearly 6,000 flights a day this time of year.
Delta's travel waiver has been extended. The airline has extended its offer of compensation to customers significantly affected by lengthy delays or cancellations. Many customers have already been contacted by Delta.
---
12:30 PM ET // August 10th, 2016 - Delta has added a few more cancellations, bringing the number today to 278. About 1,950 flights have departed, 75 percent of those within 30 minutes of scheduled departure time. Delta operates nearly 6,000 flights a day this time of year.
Delta also extended its offer of compensation to customers significantly affected by delays or lengthy cancellations. Many customers have already been contacted by Delta.
---
11:00AM EDT // August 10th, 2016 - Delta officials anticipate returning to a normal operation by afternoon Wednesday following an outage Monday that disrupted the flight schedule, though scattered thunderstorms expected in the eastern U.S. may slow the recovery.
By late morning, Delta had canceled 255 flights while 1,500 flights departed. About 80 percent of those flights departed within 30 minutes of their scheduled time.
Technology systems that allow airport customer service agents to process check-ins, conduct boarding and dispatch aircraft are functioning normally with the bulk of delays and cancellations coming as a result of flight crews displaced or running up against their maximum allowed duty period following the outage.
Were in the final hours of bouncing back from the disruption, said Bill Lentsch, Deltas Senior Vice President Airport Customer Service and Airline Operations. Delta employees have been working around the clock and are committed to bringing the airline back to full strength. We know this has been a rough couple of days for our customers and apologize to those who have experienced our less-than-stellar operation.
Leaders and employees continue to focus on promptly moving departures at the airline's Atlanta hub, the world's busiest. Delta sent reservations personnel from the corporate headquarters in Atlanta to help customer service agents process customers at Hartsfield-Jackson Atlanta International Airport. A cadre of additional employees were also positioned throughout the Atlanta airport to help customers get to their flights.
Customers traveling today should check the status of their flight at delta.com or the Fly Delta App. Customers can rebook their flight via the website.
Delta's travel waiver has been extended and the airline continued to provide hotel vouchers to several thousand customers, including more than 2,300 Tuesday night in Atlanta. Delta extended its offer of compensation to customers significantly affected by delays or cancellations. Many customers have already been contacted by Delta.
---
7:00AM ET // August, 10th, 2016 - Delta officials anticipate returning to a normal operation by mid-to-late afternoon Wednesday following an outage Monday that disrupted the airlines flight schedule, though a chance of scattered thunderstorms expected in the eastern U.S. may have the potential to slow the recovery.
Teams across the airline continued work during the overnight hours to bring an end to the impact of Mondays outage that disrupted operations. Delta will start the Wednesday operation with a little over 150 system-wide cancellations after roughly 800 flights were canceled Tuesday.
Technology systems that allow airport customer service agents to process check-ins, conduct boarding and dispatch aircraft are functioning normally with the bulk of delays and cancellations coming as a result of flight crews displaced or running up against their maximum allowed duty period following the outage.
Were in the final hours of bouncing back from the disruption, said Bill Lentsch, Deltas Senior Vice President Airport Customer Service and Airline Operations. Delta employees have been working around the clock and are committed to bringing the airline back to full strength. We know this has been a rough couple of days for our customers and apologize to those who have experienced our less-than-stellar operation.
Delta CEO Bastian: 'This isn't who we are'
Leaders and employees continue to focus on promptly moving departures at the airline's Atlanta hub, the world's busiest. Delta sent reservations personnel from the corporate headquarters in Atlanta to help customer service agents process customers at Hartsfield-Jackson Atlanta International Airport. A cadre of additional employees were also positioned throughout the Atlanta Airport to help customers get to their flights.
Customers traveling today should check the status of their flight at delta.com or the Fly Delta App. Customers can rebook their flight via the website.
Delta's travel waiver has been extended and the airline continued to provide hotel vouchers to several thousand customers, including more than 2,300 Tuesday night in Atlanta. Through noon ET, Delta will extend its offer of compensation to customers significantly affected by delays or cancellations to cover Tuesday. Many customers have already been contacted by Delta agents.
Unaccompanied minors will again be allowed to travel on Delta Wednesday following a temporary stay during the height of the disruption.
---
August 9th, 2016 // 6:00 PM EST - Rebooking, Refunds, and Compensation: What Customers Need to Know:
Delta people worldwide are working around the clock to take care of customers affected by the disruption of airline operations. Below are updates to assist customers with questions regarding rebooking a flight, the current travel waiver and compensation Delta is offering affected customers:
Q: What does Delta's travel waiver offer customers?
If a customer's flight is cancelled or significantly delayed, he or she entitled to a refund. Even if the flight is not canceled, a one-time change to the ticket may be made without a fee. See full details here.
Q. Has Delta's travel waiver been extended?
Yes, to offer additional flexibility to customers who wish to make changes to their travel plans, Delta's travel waiver has been extended. For the latest information, see full details here.
Q. Is Delta offering travel vouchers to impacted customers?
Yes. Delta will provide $200 in travel vouchers to all customers who experienced a delay of greater than three hours or a cancelled flight as a result of the impact to operations systemwide. The vouchers are available for travel on all Delta and Delta Connection-operated flights.
Customers affected by the outage and who have provided their email address will be contacted via email and will receive their voucher in seven to 10 business days. Customers who have not been contacted can complete a Customer Care form on www.delta.com/wecare to receive their voucher. All travel must be booked by one year from the date the voucher was issued.
Q. Do the travel vouchers also apply to customers who are entitled to a refund?
Yes. Delta will provide $200 in travel vouchers to all customers who experienced a delay of greater than three hours or a cancelled flight as a result of the impact to operations systemwide.
---
August 9th, 2016 - Delta CEO Ed Bastian Releases Another Statement
Today Ed released another statement regarding the ongoing system reset and outage. Claiming "This isn't who we are". It can be found here.
---
Nonrev Travel Discouraged; Buddy Passes embargoed
Delta employees and family with nonrev flight privileges are discouraged at this time from making any personal travel until further notice. Additionally, during this time travelling via the use of a buddy pass will be embargoed. An end date to this travel suspension has not been decided upon and will be released in future updates.
---
Travel Waiver in effect:
All Delta flights
August 8-10, 2016
Ticket must be Reissued On or Before, and Rebooked Travel must begin no later than:
August 21, 2016
Note: This also has been extended to include hotel rooms and other accommodations for those customers who may need to take an add'l night stay due to a flight cancellation.
---
August 9th, 2016 // 11:00 PM EST -
Delta canceled about 775 flights Tuesday as the airline worked to establish normal operations after Mondays outage that disrupted the flight schedule. The outlook for Wednesdays operation is positive, Delta officials said, with about 90 cancellations expected at the beginning of the day leading to normal operations later.
Delta continues to recover from events earlier this week and expect to be back to normal operations by mid-day Wed, with some cancellations in the morning. -summarized; news.delta.com
---
August 9th, 2016 // 2:00 PM EST - Cause of Outage Identified:
"a critical power control module at our Technology Command Center malfunctioned, causing a surge to the transformer and a loss of power. The universal power was stabilized and power was restored quickly. But when this happened, critical systems and network equipment didn't switch over to backups. Other systems did. And now we're seeing instability in these systems."
Power was restored quickly however, due to some critical systems and network equipment didn't switch over to the backups leading to the outage; DL still in recovery. ACS systems are still catching up and have been reported to be running slow -summarized; news.delta.com
---
August 9th, 2016 // 12:30 PM EST - Delta has canceled about 530 flights today while more than 1,600 have departed as the airline works diligently to normalize operations after Monday's outage that disrupted the flight schedule. Leaders and employees continue to focus on promptly moving departures at the airline's Atlanta hub, the world's busiest.
Customers traveling today should check the status of their flight at delta.com or the Fly Delta App. Customers can rebook their flight via the website.
Delta's travel waiver has been extended to customers traveling on Aug. 9. Also, unaccompanied minors that have not yet begun travel will not be accepted until Wednesday. Those customers will be able to book without fee at a later date.
Delta offered compensation to customers significantly affected by delays or cancellations. -news.delta.com
---
August 9th, 2016 // 8:30 AM EST - What Customers Need to Know for Tuesday:
Delta is cancelling more than 300 flights Tuesday morning as it works to reset the operation and get crews, aircraft and other operational elements in place to take care of customers. The airline experienced a loss of power Monday morning that impacted operations systemwide.
Here are some things to know today:
1. Customers traveling Tuesday should check the status of their flight at delta.com or the Fly Delta App. Customers can rebook their flight via the website.
2. Delays continue. Customers heading to the airport should expect delays and cancellations. While inquiries are high and wait times are long, our customer service agents are doing everything they can to assist.
3. Delta's travel waiver has been extended to customers traveling on Tuesday.
4. Delta extended its offer of compensation to customers significantly affected by delays or cancellations to cover today.
5. Unaccompanied minors that have not yet begun travel will not be accepted until Wednesday. Those customers will be able to book without fee at a later date.-news.delta.com
You may all feel free to edit it as your leisure as we continue to watch this develop..
If you are currently at an airport right now, your information on the ongoing events and current situation there is equally as valuable. Thanks for your input.
If you have part of an article in your post, please cite it with a URL to it's
origin.
Airport & Support Situation..
NRT: Foam or Blowup Pad, Crackers, and water given out to customers for sleeping prior to hotel voucher last night
MSP: "Security Lines best since reconfiguration"
---
Note:
Some passengers are reporting long waiting queues on DL phone lines. If possible avoid contacting DL CSR by phone (DM and medallion lines are also expected to have a notably longer wait as well) and instead redirect att'n to Twitter which has been said to be notably quicker than other means of assistance, including Need Help Centers and Gate Agents. Fly Delta and Delta.com should also be permitting rebooking on to other flights at this time. If you are unable to take advantage of any of these methods, express add'l patience
Delta Related Updates:
---
August 11th, 2016 // 11:00AM ET - Flights OPs Return to Normal
Flight operations have returned to normal following Mondays outage, airline officials said at noon today. Delta started the day with roughly 25 mainline flights canceledsome due to bad weather in the Midwestand no additional cancellations are anticipated.
The airline is back on its feet, moving past this weeks disruption and operating normally, said Dave Holtz, S.V.P. - Operations and Customer Center. Running a reliable operation is something we pride ourselves on and frankly, that didnt happen this week. But our teams have worked tirelessly to bring back the dependable airline our customers have known us to be and have come to expect from us.
SLC.jpgWith the impact of the power loss is in the rearview mirror, the airline is carefully monitoring the development of a weather system that may hit the eastern third of the U.S. later today, with the possibility of air traffic control-related delays possible in the Northeast. No additional cancellations as a result of weather are expected, however.
The airline plans to fly more than 610,000 people today. By noon Thursday, nearly 2,200 flights had departed with more than 85 percent departing within 15 minutes of the scheduled departure time.
Even so, customers traveling today should check the status of their flight at delta.com or the Fly Delta App. Delta's travel waiver has been extended through Thursday.
Monday morning an uninterrupted power source switch experienced a small fire which resulted in a massive failure at Deltas Technology Command Center. This caused the power control module to malfunction, sending a surge to a transformer outside of Delta, resulting in the loss of power. The power was stabilized and power was restored quickly. But when this happened, critical systems and network equipment didnt switch over to backups. Around 300 of about 7,000 data center components were discovered to not have been configured appropriately to avail back-up power. In addition to restoring Deltas systems to normal operations, Delta teams this week have been working to ensure reliable redundancies of electrical power as well as network connectivity and applications are in place.
---
August 10th, 2016
Travel waiver impacted dates now extended to include August 11th
https://www.delta.com/content/www/en...em-outage.html
---
4:30PM ET // August 10th, 2016 - Delta has canceled 317 flights today, none in the last hour. Of 3,100 flights that have departed during the day, about 2/3 left within 30 minutes of their scheduled departure time.
Delta's travel waiver has been extended. The airline has extended its offer of compensation to customers significantly affected by lengthy delays or cancellations. Many customers have already been contacted by Delta.
Delta CEO Bastian: 'This isn't who we are'
---
2:30PM ET // August 10th, 2016 - Delta has canceled 311 flights today. About 2,540 flights have departed, 70 percent of those within 30 minutes of the scheduled departure time. Delta operates nearly 6,000 flights a day this time of year.
Delta's travel waiver has been extended. The airline has extended its offer of compensation to customers significantly affected by lengthy delays or cancellations. Many customers have already been contacted by Delta.
---
12:30 PM ET // August 10th, 2016 - Delta has added a few more cancellations, bringing the number today to 278. About 1,950 flights have departed, 75 percent of those within 30 minutes of scheduled departure time. Delta operates nearly 6,000 flights a day this time of year.
Delta also extended its offer of compensation to customers significantly affected by delays or lengthy cancellations. Many customers have already been contacted by Delta.
---
11:00AM EDT // August 10th, 2016 - Delta officials anticipate returning to a normal operation by afternoon Wednesday following an outage Monday that disrupted the flight schedule, though scattered thunderstorms expected in the eastern U.S. may slow the recovery.
By late morning, Delta had canceled 255 flights while 1,500 flights departed. About 80 percent of those flights departed within 30 minutes of their scheduled time.
Technology systems that allow airport customer service agents to process check-ins, conduct boarding and dispatch aircraft are functioning normally with the bulk of delays and cancellations coming as a result of flight crews displaced or running up against their maximum allowed duty period following the outage.
Were in the final hours of bouncing back from the disruption, said Bill Lentsch, Deltas Senior Vice President Airport Customer Service and Airline Operations. Delta employees have been working around the clock and are committed to bringing the airline back to full strength. We know this has been a rough couple of days for our customers and apologize to those who have experienced our less-than-stellar operation.
Leaders and employees continue to focus on promptly moving departures at the airline's Atlanta hub, the world's busiest. Delta sent reservations personnel from the corporate headquarters in Atlanta to help customer service agents process customers at Hartsfield-Jackson Atlanta International Airport. A cadre of additional employees were also positioned throughout the Atlanta airport to help customers get to their flights.
Customers traveling today should check the status of their flight at delta.com or the Fly Delta App. Customers can rebook their flight via the website.
Delta's travel waiver has been extended and the airline continued to provide hotel vouchers to several thousand customers, including more than 2,300 Tuesday night in Atlanta. Delta extended its offer of compensation to customers significantly affected by delays or cancellations. Many customers have already been contacted by Delta.
---
7:00AM ET // August, 10th, 2016 - Delta officials anticipate returning to a normal operation by mid-to-late afternoon Wednesday following an outage Monday that disrupted the airlines flight schedule, though a chance of scattered thunderstorms expected in the eastern U.S. may have the potential to slow the recovery.
Teams across the airline continued work during the overnight hours to bring an end to the impact of Mondays outage that disrupted operations. Delta will start the Wednesday operation with a little over 150 system-wide cancellations after roughly 800 flights were canceled Tuesday.
Technology systems that allow airport customer service agents to process check-ins, conduct boarding and dispatch aircraft are functioning normally with the bulk of delays and cancellations coming as a result of flight crews displaced or running up against their maximum allowed duty period following the outage.
Were in the final hours of bouncing back from the disruption, said Bill Lentsch, Deltas Senior Vice President Airport Customer Service and Airline Operations. Delta employees have been working around the clock and are committed to bringing the airline back to full strength. We know this has been a rough couple of days for our customers and apologize to those who have experienced our less-than-stellar operation.
Delta CEO Bastian: 'This isn't who we are'
Leaders and employees continue to focus on promptly moving departures at the airline's Atlanta hub, the world's busiest. Delta sent reservations personnel from the corporate headquarters in Atlanta to help customer service agents process customers at Hartsfield-Jackson Atlanta International Airport. A cadre of additional employees were also positioned throughout the Atlanta Airport to help customers get to their flights.
Customers traveling today should check the status of their flight at delta.com or the Fly Delta App. Customers can rebook their flight via the website.
Delta's travel waiver has been extended and the airline continued to provide hotel vouchers to several thousand customers, including more than 2,300 Tuesday night in Atlanta. Through noon ET, Delta will extend its offer of compensation to customers significantly affected by delays or cancellations to cover Tuesday. Many customers have already been contacted by Delta agents.
Unaccompanied minors will again be allowed to travel on Delta Wednesday following a temporary stay during the height of the disruption.
---
August 9th, 2016 // 6:00 PM EST - Rebooking, Refunds, and Compensation: What Customers Need to Know:
Delta people worldwide are working around the clock to take care of customers affected by the disruption of airline operations. Below are updates to assist customers with questions regarding rebooking a flight, the current travel waiver and compensation Delta is offering affected customers:
Q: What does Delta's travel waiver offer customers?
If a customer's flight is cancelled or significantly delayed, he or she entitled to a refund. Even if the flight is not canceled, a one-time change to the ticket may be made without a fee. See full details here.
Q. Has Delta's travel waiver been extended?
Yes, to offer additional flexibility to customers who wish to make changes to their travel plans, Delta's travel waiver has been extended. For the latest information, see full details here.
Q. Is Delta offering travel vouchers to impacted customers?
Yes. Delta will provide $200 in travel vouchers to all customers who experienced a delay of greater than three hours or a cancelled flight as a result of the impact to operations systemwide. The vouchers are available for travel on all Delta and Delta Connection-operated flights.
Customers affected by the outage and who have provided their email address will be contacted via email and will receive their voucher in seven to 10 business days. Customers who have not been contacted can complete a Customer Care form on www.delta.com/wecare to receive their voucher. All travel must be booked by one year from the date the voucher was issued.
Q. Do the travel vouchers also apply to customers who are entitled to a refund?
Yes. Delta will provide $200 in travel vouchers to all customers who experienced a delay of greater than three hours or a cancelled flight as a result of the impact to operations systemwide.
---
August 9th, 2016 - Delta CEO Ed Bastian Releases Another Statement
Today Ed released another statement regarding the ongoing system reset and outage. Claiming "This isn't who we are". It can be found here.
---
Nonrev Travel Discouraged; Buddy Passes embargoed
Delta employees and family with nonrev flight privileges are discouraged at this time from making any personal travel until further notice. Additionally, during this time travelling via the use of a buddy pass will be embargoed. An end date to this travel suspension has not been decided upon and will be released in future updates.
---
Travel Waiver in effect:
All Delta flights
August 8-10, 2016
Ticket must be Reissued On or Before, and Rebooked Travel must begin no later than:
August 21, 2016
Note: This also has been extended to include hotel rooms and other accommodations for those customers who may need to take an add'l night stay due to a flight cancellation.
---
August 9th, 2016 // 11:00 PM EST -
Delta canceled about 775 flights Tuesday as the airline worked to establish normal operations after Mondays outage that disrupted the flight schedule. The outlook for Wednesdays operation is positive, Delta officials said, with about 90 cancellations expected at the beginning of the day leading to normal operations later.
Delta continues to recover from events earlier this week and expect to be back to normal operations by mid-day Wed, with some cancellations in the morning. -summarized; news.delta.com
---
August 9th, 2016 // 2:00 PM EST - Cause of Outage Identified:
"a critical power control module at our Technology Command Center malfunctioned, causing a surge to the transformer and a loss of power. The universal power was stabilized and power was restored quickly. But when this happened, critical systems and network equipment didn't switch over to backups. Other systems did. And now we're seeing instability in these systems."
Power was restored quickly however, due to some critical systems and network equipment didn't switch over to the backups leading to the outage; DL still in recovery. ACS systems are still catching up and have been reported to be running slow -summarized; news.delta.com
---
August 9th, 2016 // 12:30 PM EST - Delta has canceled about 530 flights today while more than 1,600 have departed as the airline works diligently to normalize operations after Monday's outage that disrupted the flight schedule. Leaders and employees continue to focus on promptly moving departures at the airline's Atlanta hub, the world's busiest.
Customers traveling today should check the status of their flight at delta.com or the Fly Delta App. Customers can rebook their flight via the website.
Delta's travel waiver has been extended to customers traveling on Aug. 9. Also, unaccompanied minors that have not yet begun travel will not be accepted until Wednesday. Those customers will be able to book without fee at a later date.
Delta offered compensation to customers significantly affected by delays or cancellations. -news.delta.com
---
August 9th, 2016 // 8:30 AM EST - What Customers Need to Know for Tuesday:
Delta is cancelling more than 300 flights Tuesday morning as it works to reset the operation and get crews, aircraft and other operational elements in place to take care of customers. The airline experienced a loss of power Monday morning that impacted operations systemwide.
Here are some things to know today:
1. Customers traveling Tuesday should check the status of their flight at delta.com or the Fly Delta App. Customers can rebook their flight via the website.
2. Delays continue. Customers heading to the airport should expect delays and cancellations. While inquiries are high and wait times are long, our customer service agents are doing everything they can to assist.
3. Delta's travel waiver has been extended to customers traveling on Tuesday.
4. Delta extended its offer of compensation to customers significantly affected by delays or cancellations to cover today.
5. Unaccompanied minors that have not yet begun travel will not be accepted until Wednesday. Those customers will be able to book without fee at a later date.-news.delta.com
Delta computers down
Aug 9, 2016 | 6:41 am
#511
FlyerTalk Evangelist
Join Date: Sep 2003
Location: San Antonio
Programs: DL DM, Former AA EXP now AY Plat, AC 75K, NW Plat, Former CO Gold, Hilton Diamond, Marriott Titanium
Posts: 28,002
DL Connection aren't effected. They don't depend on DL for operations. It's done by the carrier itself.
Aug 9, 2016 | 6:41 am
#512
Join Date: Jun 2014
Location: Madison, AL
Posts: 195
I too ran into trouble at HSV. My wife and I were scheduled to travel to AUS via ATL this morning departing at 6:40am. We got a notice on the way to the airport that I was rebooked on the 5:25am flight and would arrive at 11:49pm tonight with a connection at ATL and MSP. She was rebooked on United departing around 6am through Houston and arriving this morning, but we arrived too late at the airport for her to make that flight. I have business in Austin on Wed afternoon and all day Thursday and we were only going a day early to spend time with family there. We canceled her flights and supposedly with receive a full refund plus a $200.00 voucher. I changed my flight until tomorrow morning and was also promised a $200.00 voucher. Now I am trying to check into my flight tomorrow and am getting a notice that the ATL to AUS segment is overbooked and if I would be willing to take a later flight. Tomorrow might not be so much fun either.
Aug 9, 2016 | 6:41 am
#513
Join Date: Dec 2003
Location: MEM
Programs: DL DM, Marriott/Starwood Plat
Posts: 441
I'd agree with many of your statements except that there are 1000's of busineses today that are performing the same logic on vastly superior systems. The idea of single state is NOT new or even airline centric. You're got to be kidding yourself if you think the airline industry is THAT unique. This isn't new or ground braking.
Want an example in the airline industry? How do GDS's some distribute inventory and fare invormation with any (that I know of) interruption in many many years?
Distriubuted mainframes. It's possible. Let's stop this idea of delta is smarter than IT. they aren't. It's proven. They need better people (no more dissimilar than IT organization when this happens).
Want an example in the airline industry? How do GDS's some distribute inventory and fare invormation with any (that I know of) interruption in many many years?
Distriubuted mainframes. It's possible. Let's stop this idea of delta is smarter than IT. they aren't. It's proven. They need better people (no more dissimilar than IT organization when this happens).
If it was as easy as putting it in the cloud the airlines and banks would have done it. If you are so convinced that the problem of stateful active active over high latency for sequential immediately consistent transactions is trivial you should start a software company.
Aug 9, 2016 | 6:53 am
#514
Join Date: Jul 2010
Programs: Marriott Titanium, DL DM
Posts: 298
From my perspective those who got to sleep in the airport are lucky. My flight was continually pushed back and moved until they finally cancelled at 3 AM (9pm original). Then I was on hold over amy hour with diamond desk to rebook because the app wouldn't let me rebook and as far as it said I had no flight scheduled.
Finally made it out, missing my first meeting and getting no sleep.
Finally made it out, missing my first meeting and getting no sleep.
Aug 9, 2016 | 7:02 am
#515
Join Date: Mar 2013
Location: US of A
Programs: Delta Diamond, United 1K, BA Blue, Marriott Titanium, Hilton Gold, Amex Platinum
Posts: 1,786
Industries that provide such crucial service cannot be left to regulate themselves with respect to pax compensation and minimum standards of service when it comes to IRROPS. History proves that it is not long before other carriers follow suit and it becomes a race to the bottom. I am not going to hypothesise what it may or may not lead to, but airlines will always compete on price. Airlines will always try to do something on the cheap in order to make more profit, so having legislation imposing minimum compensation and duty of care requirements would provide that safety net for pax that they deserve.
Aug 9, 2016 | 7:07 am
#516
Join Date: Nov 2013
Programs: DL PM
Posts: 1,089
The GDSs don't hold the inventory, they are connected to the airline reservations systems and when you book with the GDSs it writes your reservation to the same monolithic airline reservation application on the back end we have been talking about being down all day.
If it was as easy as putting it in the cloud the airlines and banks would have done it. If you are so convinced that the problem of stateful active active over high latency for sequential immediately consistent transactions is trivial you should start a software company.
If it was as easy as putting it in the cloud the airlines and banks would have done it. If you are so convinced that the problem of stateful active active over high latency for sequential immediately consistent transactions is trivial you should start a software company.
1) DB failover: redundant DBs that automatically replicate and self-elect a master. Requires 3 or more members so you can have a quorum during master election. During normal operations, all reads and writes happen to an appointed master instance. All instances send "are you still online?" Heartbeat messages to each other to verify status, and slaves replicate data from the master as soon as it is written. If the master ever goes offline, the remaining members vote and elect a new master, automatically resuming operations.
2) blockchain-like distributed ledger. Biggest problem I see with this I current implementations is significant time delay while transactions are verified (proof of work currently takes lots of time, by design). But on the upside, even "untusted" parties can help contribute. Doesn't make a lot of sense for an airline.
3) fully redundant DBs: similar to a raid disk array. Writes happen across multiple DBs in parallel. Writes are verified for congurancy across all mirrors when a read occurs. If a mirror drops, the set is degraded, but still remains functional until the mirror comes back online. Biggest downside to this approach is the additional time needed to verify reads, which may not scale fantastically. But then again, we're talking milliseconds... If you have to wait a few extra ms to confirm that seat/ticket/weight balance call/whatever, it's probally not the end of he world.
Aug 9, 2016 | 7:07 am
#517
Suspended
Join Date: Dec 2011
Location: RDU
Programs: DL DM, HHonors Diamond, Marriott Platinum, etc etc etc
Posts: 2,341
Honestly instead of a dumb 58 second video from the CEO, I would just like them to come out, and admit what really happened.
There were no power outages in ATL that night, unless they are on some magical grid that only goes to them. (I am sure Georgia Power is scratching their heads on that one)
Tell people what really happened, what was done to fix it, and how you will work to avoid this happening in the future. Marketing must think the current approach is better, but the truth will go a long way for DL going forward.
DR's dont always work, it takes one outdated piece of equipment in a data center to crash the whole thing, etc. just admit your mistakes DL.
There were no power outages in ATL that night, unless they are on some magical grid that only goes to them. (I am sure Georgia Power is scratching their heads on that one)
Tell people what really happened, what was done to fix it, and how you will work to avoid this happening in the future. Marketing must think the current approach is better, but the truth will go a long way for DL going forward.
DR's dont always work, it takes one outdated piece of equipment in a data center to crash the whole thing, etc. just admit your mistakes DL.
Aug 9, 2016 | 7:15 am
#518
Join Date: Mar 2005
Posts: 28
I don't think anyone is saying it is trivial... But I also don't think it needs to be that hard either. I see a few possible solutions to this problem (I am by no means an expert here):
1) DB failover: redundant DBs that automatically replicate and self-elect a master. Requires 3 or more members so you can have a quorum during master election. During normal operations, all reads and writes happen to an appointed master instance. All instances send "are you still online?" Heartbeat messages to each other to verify status, and slaves replicate data from the master as soon as it is written. If the master ever goes offline, the remaining members vote and elect a new master, automatically resuming operations.
2) blockchain-like distributed ledger. Biggest problem I see with this I current implementations is significant time delay while transactions are verified (proof of work currently takes lots of time, by design). But on the upside, even "untusted" parties can help contribute. Doesn't make a lot of sense for an airline.
3) fully redundant DBs: similar to a raid disk array. Writes happen across multiple DBs in parallel. Writes are verified for congurancy across all mirrors when a read occurs. If a mirror drops, the set is degraded, but still remains functional until the mirror comes back online. Biggest downside to this approach is the additional time needed to verify reads, which may not scale fantastically. But then again, we're talking milliseconds... If you have to wait a few extra ms to confirm that seat/ticket/weight balance call/whatever, it's probally not the end of he world.
1) DB failover: redundant DBs that automatically replicate and self-elect a master. Requires 3 or more members so you can have a quorum during master election. During normal operations, all reads and writes happen to an appointed master instance. All instances send "are you still online?" Heartbeat messages to each other to verify status, and slaves replicate data from the master as soon as it is written. If the master ever goes offline, the remaining members vote and elect a new master, automatically resuming operations.
2) blockchain-like distributed ledger. Biggest problem I see with this I current implementations is significant time delay while transactions are verified (proof of work currently takes lots of time, by design). But on the upside, even "untusted" parties can help contribute. Doesn't make a lot of sense for an airline.
3) fully redundant DBs: similar to a raid disk array. Writes happen across multiple DBs in parallel. Writes are verified for congurancy across all mirrors when a read occurs. If a mirror drops, the set is degraded, but still remains functional until the mirror comes back online. Biggest downside to this approach is the additional time needed to verify reads, which may not scale fantastically. But then again, we're talking milliseconds... If you have to wait a few extra ms to confirm that seat/ticket/weight balance call/whatever, it's probally not the end of he world.
The amount of "I can't believe they didn't have a backup system!" naivety and general speculation about something no one here clearly has knowledge of us appalling.
Can we keep all the armchair IT quarterbacking out of this until we have some sort of definitive information about what happened and have someone who can start a post with "I AM an expert"?
Aug 9, 2016 | 7:17 am
#519
Join Date: May 2015
Posts: 2,881
Honestly instead of a dumb 58 second video from the CEO, I would just like them to come out, and admit what really happened.
There were no power outages in ATL that night, unless they are on some magical grid that only goes to them. (I am sure Georgia Power is scratching their heads on that one)
Tell people what really happened, what was done to fix it, and how you will work to avoid this happening in the future. Marketing must think the current approach is better, but the truth will go a long way for DL going forward.
DR's dont always work, it takes one outdated piece of equipment in a data center to crash the whole thing, etc. just admit your mistakes DL.
There were no power outages in ATL that night, unless they are on some magical grid that only goes to them. (I am sure Georgia Power is scratching their heads on that one)
Tell people what really happened, what was done to fix it, and how you will work to avoid this happening in the future. Marketing must think the current approach is better, but the truth will go a long way for DL going forward.
DR's dont always work, it takes one outdated piece of equipment in a data center to crash the whole thing, etc. just admit your mistakes DL.
Aug 9, 2016 | 7:18 am
#520
Join Date: Jul 2005
Location: National Capitol Region
Programs: Delta Dirt Medallion,AA,USairways, WN Rapid Rewards, National Emerald Club
Posts: 3,913
Honestly instead of a dumb 58 second video from the CEO, I would just like them to come out, and admit what really happened.
There were no power outages in ATL that night, unless they are on some magical grid that only goes to them. (I am sure Georgia Power is scratching their heads on that one).
There were no power outages in ATL that night, unless they are on some magical grid that only goes to them. (I am sure Georgia Power is scratching their heads on that one)
.http://www.nbcnews.com/business/trav...ailure-n625806
Aug 9, 2016 | 7:23 am
#521
Join Date: Oct 2007
Programs: Now just a lowly DL PM/1MM. This industry needs some competition. It's just not enjoyable anymore.
Posts: 3,545
No it's not scratching its head, it would seem Georgia power already pointed the finger at Delta.
http://www.nbcnews.com/business/trav...ailure-n625806
http://www.nbcnews.com/business/trav...ailure-n625806
Not traveling until Saturday. I sincerely hope this mess is resolved by then.
Aug 9, 2016 | 7:27 am
#522
FlyerTalk Evangelist
Join Date: Sep 2003
Location: San Antonio
Programs: DL DM, Former AA EXP now AY Plat, AC 75K, NW Plat, Former CO Gold, Hilton Diamond, Marriott Titanium
Posts: 28,002
Honestly instead of a dumb 58 second video from the CEO, I would just like them to come out, and admit what really happened.
There were no power outages in ATL that night, unless they are on some magical grid that only goes to them. (I am sure Georgia Power is scratching their heads on that one)
Tell people what really happened, what was done to fix it, and how you will work to avoid this happening in the future. Marketing must think the current approach is better, but the truth will go a long way for DL going forward.
DR's dont always work, it takes one outdated piece of equipment in a data center to crash the whole thing, etc. just admit your mistakes DL.
There were no power outages in ATL that night, unless they are on some magical grid that only goes to them. (I am sure Georgia Power is scratching their heads on that one)
Tell people what really happened, what was done to fix it, and how you will work to avoid this happening in the future. Marketing must think the current approach is better, but the truth will go a long way for DL going forward.
DR's dont always work, it takes one outdated piece of equipment in a data center to crash the whole thing, etc. just admit your mistakes DL.
Fixing it to ensure you have enough power for a data center in case of power failure is not easy. Older DCs weren't designed with the power consumption backup that today's DCs need. Upgrading is neither cheap, nor easy. I worked at a DC providing support for 5 hospitals and all associated systems. At the time we upgraded power backups, they only could produce 70% of needed power. This was a data center that only ten years ago had power backup upgraded.
The speed at which technology has come into play has vastly exceeded that which data centers wouldve needed and been designed for only a few years before. Every server, every router, every computer or other electronic adds to the power demand. To replace those backup systems in general means taking the whole system down. In the case at the data center I worked that meant being without any computer technology for nearly 12 hours for 5 hospitals. Mind you this is a small data center. The likes of DL'S would be many times bigger.
As for the reason, a wait and see is best in cases like this. At this stage everyone wants to not be the one responsible and will put out PR trying to touted that. Wait a week and we'll have a clearer idea what caused this to start.
Last edited by flyerCO; Aug 9, 2016 at 7:37 am
Aug 9, 2016 | 7:27 am
#523
Suspended
Join Date: Dec 2011
Location: RDU
Programs: DL DM, HHonors Diamond, Marriott Platinum, etc etc etc
Posts: 2,341
No it's not scratching its head, it would seem Georgia power already pointed the finger at Delta.
http://www.nbcnews.com/business/trav...ailure-n625806
http://www.nbcnews.com/business/trav...ailure-n625806
Aug 9, 2016 | 7:28 am
#524
Suspended
Join Date: Dec 2011
Location: RDU
Programs: DL DM, HHonors Diamond, Marriott Platinum, etc etc etc
Posts: 2,341
I slightly disagree with you, DL is putting out the BS in hopes that the answer will be "good enough" and people will forget.