FlyerTalk Forums - View Single Post - LH Global IT system outage 15FEB2023
View Single Post
Old Feb 16, 2023 | 10:49 pm
  #55  
KRSW
10 Countries Visited
20 Countries Visited
30 Countries Visited
15 Years on Site
 
Join Date: Aug 2008
Location: Somewhere in Florida
Posts: 2,884
Seriously, how does something like this happen? I work for a small business (10 employees) and even we have redundant connections using different technologies; Comcast, a microwave link on the roof to a data center down the street, and a cellular modem as a last resort. I don't need to test the links, as Comcast goes down regularly (at least 1-3x a month), and the dual HA routers automatically re-route the traffic with no fuss. 26 years strong despite many hurricanes and tornadoes, and even a drunk driver crashing into the building. Difficulty: We can't use cloud services/servers because contractual agreements with some of our clients.

Our first failure was last year when Hurricane Ian ripped the roof of the data center cleanly off, flooding the racks and destroying wiring and antennas. The rain killed our e-mail server. Everything else kept going, and we were down to the cell modem for reaching the outside world. 6 hours later the e-mail server was running again and they'd managed to rig up a temporary link to the data center. Comcast was down for weeks. Again, this is a 10-person office with no budget, running absolute garbage equipment (circa-2003 Dell PowerEdge 2650s). How come Lufty with €16.81 billion in revenue can't do this?

SLAs are useless. Great, you'll get compensated 1 day of service for every hour you're down. That doesn't stop your connection from going down. SLAs also have exclusions for fiber-eating-backhoes and natural disasters.
KRSW is offline