No dispute there-getting angry in the heat of it doesn't help-I've been there before (albeit with a much smaller company than DL) and that's exactly what you do-get things working and then establish what exactly happened and remediate so you can avoid that in the future. It just wouldn't be shocking for the "remediate" to involve personnel changes.

These systems are never supposed to just lose power ever, and they are never supposed to all just be off at the same time ever.

So yeah, totally plausible... They have to bring the systems back up, probably in a certain order, and make sure they're talking to each other right.

This sort of thing is just never supposed to happen so recovery is likely very manual and slow.

I would put some money on storage fabric failure. Was with a "task force" to repair that at a large german car maker -- took two days+nights w/ some 50 people. And yes, heads rolled later, not upfront.

I've seen it happen before so it is possible, but did someone forget to fuel up the generators? :confused:

Thankfully I'm not flying anywhere this week.

Maybe they forgot to feed the mice so that at some point those guys just stopped running in their little wheels to generate the power.

VDB
 

I'm sure many on FT are considering this right now but what do you guys think about grabbing a refundable ticket and heading to the airport for a possible VDB?

I've seen stranger, but it sounds unlikely. Typically you test the generators at a regular interval so you have a refuel contract (assuming you're not on a natural gas line which I've personally never been). They come and top off the tanks because even the generator test will use some amount of NatGas.

The more plausible answer (given that we're all on the outside) is a battery failure. The generator can't kick on the second you lose power, so the batteries provide a bridge, if you will. Generally the batteries are only rated for a few minutes (as in design, they would only be used for seconds). However, something that can happen is from years and years of sitting there (or other reasons), the batteries don't have as much power so basically, the bridge collapses between utility power and generator power, and thus you have a power failure.

I just purchased 5 tickets and called in and they are sending me $300 per a ticket since I am delayed. $1,500 nice :)

In my example, the architect would probably be the one getting canned for bad guidance actually.

In all reality, everything can be redundant, simply put. Knowing only that Deltamatic is IBM zSeries or pSeries based, it's hard for me to believe that indeed it wasn't an outside system that caused the failure.

I hope they are transparent in what happened (even if it is something stupid).

Sleaze

I see more and more flights from foreign stations taking off. But still nothing mainline from nearby MSP. A couple Endeavour CRJs have departed.

Great news, Deltamatic is back up and running!

http://i64.tinypic.com/xo2oe8.png

Not surprising. They need the once-a-day jumbos moving and they also have the most time before arrival to get gates cleared etc.

Flights that land an hour after takeoff they may not be ready to deal with yet.

Haven't seen that prompt in ages. HIMEM.exe was the WORST!

Hope DL handles the recovery better than Southwest.