Community
Wiki Posts
Search

T5 - systems down, unable to board

Thread Tools
 
Search this Thread
 
Old Oct 3, 2019, 2:51 pm
  #16  
 
Join Date: Nov 2018
Location: Belfast
Programs: BA Silver
Posts: 379
Originally Posted by Tobias-UK
I suspect given the lack of activity on here the matter must have been a temporary blip and all ops back to normal.
Delay for me tonight, due to depart 1805 but didn’t get away until 1930. Captain gave reason for delay as IT issues preventing aircraft from pushing back. This in turn delayed our inbound by almost an hour as it waited on the ground for a stand to be free.

Having just read this thread, I now wonder if there was a secondary issue this evening?
SHT88T is offline  
Old Oct 3, 2019, 4:11 pm
  #17  
 
Join Date: Dec 2011
Location: DWC
Programs: OWS, *A G
Posts: 626
Boarding and departure delayed for BA1466 after 8pm, captain apologised for IT issues also affecting the flight deck (?).

They also changed our A11 gate to A4 (apparently due to the IT issue), without any notifications coming through the app, causing me to perform the sprint of my life lol.
zat_dude is offline  
Old Oct 3, 2019, 4:36 pm
  #18  
Hilton Contributor Badge
 
Join Date: Sep 2014
Location: London, UK
Programs: Virtuoso, FSPP, Hyatt Prive, Hilton Impressario, Marriott Stars/Luminous, Roswood Elite, MO Fan Club
Posts: 560
Had a delayed flight to INV this afternoon due to systems issue

Thankfully made up for lost time in the air and my B2B trip went pretty smooth
kosy91 is offline  
Old Oct 3, 2019, 5:44 pm
  #19  
Moderator: British Airways Executive Club
 
Join Date: Nov 2010
Location: TPA/ABZ
Programs: BA Lifetime Gold. GGL/CCR.
Posts: 13,248
Originally Posted by Scots_Al
Fly broken again? What a joke BA’s IT buyers are.
Originally Posted by 13901
Without solutionising, if JFE is down as well then it ain’t a problem caused by BA. FLY relies on the SIP middleware and both are BA’s; if JFE is down too then it’s an Amadeus outage for which BA can’t do much...
Hey, stop that 13901, we're not interested in FACTS around here. Let the BA bashing continue
wrp96 and 13901 like this.
golfmad is offline  
Old Oct 3, 2019, 6:34 pm
  #20  
FlyerTalk Evangelist
 
Join Date: Feb 2000
Location: London, UK and Southern France
Posts: 18,364
Originally Posted by golfmad
Hey, stop that 13901, we're not interested in FACTS around here. Let the BA bashing continue
Yes, only a BA-hater could possibly suggest that there might be anything less than perfect in BA IT systems. We all know the colossal sums that BA has invested to ensure that its IT systems are the reference in the airline industry and the envy of all its competitors.

The proposition that BA might be any problems with its IT systems at LHR seems to laughably implausible that only the usual gang of BA-bashers could possibly think something like that.
nancypants likes this.
NickB is offline  
Old Oct 3, 2019, 6:40 pm
  #21  
Moderator: British Airways Executive Club
 
Join Date: Nov 2010
Location: TPA/ABZ
Programs: BA Lifetime Gold. GGL/CCR.
Posts: 13,248
Originally Posted by NickB
Yes, only a BA-hater could possibly suggest that there might be anything less than perfect in BA IT systems. We all know the colossal sums that BA has invested to ensure that its IT systems are the reference in the airline industry and the envy of all its competitors.

The proposition that BA might be any problems with its IT systems at LHR seems to laughably implausible that only the usual gang of BA-bashers could possibly think something like that.
I see you are fluent in sarcasm as well
golfmad is offline  
Old Oct 3, 2019, 6:46 pm
  #22  
FlyerTalk Evangelist
 
Join Date: Feb 2000
Location: London, UK and Southern France
Posts: 18,364
Originally Posted by golfmad
I see you are fluent in sarcasm as well
Yes. We share a common language.
NickB is offline  
Old Oct 3, 2019, 6:52 pm
  #23  
 
Join Date: Jun 2008
Location: GLA
Programs: BA Silver
Posts: 2,962
Originally Posted by RockyRobin
Eh you mean how bad are BA's Technology staff at defining their specification and non-functional requirements

Love from an IT Buyer
Of course it comes down to specifications, but whilst the tech teams might not have the word “procurement” in their job title, they are inevitably part of the IT buying team (or should be!). Procurement people will no doubt handle the commercials, but the tech people should be the major part of the specification and technical evaluation.
Scots_Al is offline  
Old Oct 3, 2019, 9:42 pm
  #24  
 
Join Date: Jun 2015
Posts: 108
Originally Posted by Scots_Al
Of course it comes down to specifications, but whilst the tech teams might not have the word “procurement” in their job title, they are inevitably part of the IT buying team (or should be!). Procurement people will no doubt handle the commercials, but the tech people should be the major part of the specification and technical evaluation.
Yes, very valid points. But often the reality is that the procurement process is often to reduce costs, so the existing IT staff will be reduced or removed, and so they are explicitly excluded. And since the procurement people don't understand what IT does in detail, they are often sold a pup by their suppliers who are experienced at doing these procurements whereas the BA staff are not.
deep_south is offline  
Old Oct 3, 2019, 10:18 pm
  #25  
 
Join Date: Feb 2009
Location: YYC
Programs: BA bronze, Aeroplan peon
Posts: 4,746
Originally Posted by 13901
Without solutionising, if JFE is down as well then it ain’t a problem caused by BA. FLY relies on the SIP middleware and both are BA’s; if JFE is down too then it’s an Amadeus outage for which BA can’t do much...

Still sounds like a single point of failure can bring the airline to a halt, which seems like a very poor business model.
Jagboi is offline  
Old Oct 3, 2019, 11:38 pm
  #26  
 
Join Date: May 2014
Posts: 7,237
I worked on the FLY project (and the wider Travel Programme) from 2012 to 2013, leaving shortly before implementation having done my bit (don't pelt me with rotten vegetables, my bits work!!).

Scots_AI and RockyRobin mention not sufficiently-defined requirements. Having been there I can say that it wasn't the case on the software side; I mean, there have been instances where I had to explain to my business owner colleagues that "If you haven't asked for it you ain't gonna get it", but I don't believe that is fundamentally the issue with FLY. The issue, for me and for most who worked on it, is "wrong architecture" (which, yeah, can stem from wrong requirements...)

Quick explanation: Amadeus Altéa has one user interface, the JFE or Journey Front End. JFE talks with back-end systems, namely CM (Customer Management) and FM (Flight Management). Broadly speaking, one does check-in and boarding, the other weight and balances. Further up in the stream are Res and Inventory; one stores all reservations, sharing with CM those that are at -72hrs, and the other... well, it's obvious.

What BA has done was, in 2006/7, to see all that and say "Nah, we don't like the JFE". The decision was made back then to introduce a middle layer, the SIP, between back end and front end. The front end, also, would change. Enter FLY in its two flavours: FM and CM. The logic for the SIP was quite understandable at least on paper: back then JFE was a bit wobbly, though it improved considerably to the point that now 60+ major airlines use it; but also the SIP was to be the 'meeting point' of all those BA systems that consume CM and FM information, and these are legion.

The problem is that the SIP has been a sickly child for way, way too long. And it wasn't, as I understand it, an issue of wrong requirements, or TCS being not up to the job; what we were hearing was that BA was trying to be way past the bleeding edge of what was possible. Add, then, the decision of creating IAG GBS and the mass redundancies which came in 2014/15 and the cat is out of the bag.

Today I understand that BA is in the situation where it's slowly returning on its decisions, a brave thing to do in my opinion, and is rolling out the JFE to operational areas - starting with boarding and then check-in, and not just as a back-up; eventually FLY will only have limited uses. It's not as easy as it sounds because there are applications (say auto-doc-check, or conformance) that FLY and the SIP do and that Amadeus can't quite do yet.

However, where I think they stand today is that the JFE is available and largely in use; JFE has, of course, direct links to CM/FM and bypasses entirely the SIP. A SIP outage won't affect JFE. So if what happened yesterday is similar to what happened in April/May this year, when both SIP and JFE went down, then it was an Amadeus issue.

Jagboi says it's a single point of failure. I challenge that; 1A, as they call Amadeus, is an extremely stable company with a lot of redundancy. Still, sometimes it hits the fan, though only for an hour or two or so. The outage that caused a lot of grief to BA (and other 1A airlines) in April/May this year lasted, I remember, less than 120min. Having said that, I don't know of any airline that has 2 check-in systems, 2 weight-and-balance systems, 2 reservation systems and 2 inventory systems. It just cannot be done. BA used to have a back-up system for check-in, Demorel I think was its name; a very simple one, only man and a bag, and it served as a back-up for cDCS. It still took 3-4 hours to load, which made it almost useless.

Apologies for the spiel.
13901 is offline  
Old Oct 4, 2019, 12:08 am
  #27  
 
Join Date: Feb 2009
Location: YYC
Programs: BA bronze, Aeroplan peon
Posts: 4,746
Originally Posted by 13901
Jagboi says it's a single point of failure. I challenge that; 1A, as they call Amadeus, is an extremely stable company with a lot of redundancy. Still, sometimes it hits the fan, though only for an hour or two or so. The outage that caused a lot of grief to BA (and other 1A airlines) in April/May this year lasted, I remember, less than 120min. .
Thanks for the explanation! By single point of failure I meant that if Amadeus is down/unavailable/whatever, it appears that nothing moves at BA. Correct? If so, it means that everything is relying on that single data provider, so it's a "mission critical" link. Doesn't matter how good FLY is, if Amadeus is down nothing else matters. As a business, I wouldn't want everything to be relying on a single, outside provider to keep my business going, that seems like a large risk.

You've mentioned two failures/outages this year, that seems like a large number for something so important. I have friends in the telecommunications industry and I think the government mandated minimum reliability for landline phones is something like 99.9999% uptime, they called it "4 nines". I can't ever recall a time when a conventional landline was unavailable under normal circumstances ( i.e a hurricane hasn't just gone through the area). If I'm doing the calculation right, that's an outage of no more than 31 seconds per year. BA's systems don't quite seem to be at that level yet.
Scots_Al likes this.
Jagboi is offline  
Old Oct 4, 2019, 12:56 am
  #28  
 
Join Date: Jan 2011
Location: London, UK
Programs: BAGGL, A3G, Accor Gold, Hilton Diamond, IHG Diamond, LHW Sterling
Posts: 1,308
Didn’t notice any issues this morning at T5 so seems like all settled down.
Woodbinerich is offline  
Old Oct 4, 2019, 1:44 am
  #29  
FlyerTalk Evangelist
 
Join Date: Aug 2002
Location: London
Programs: Mucci. Nothing else matters.
Posts: 38,644
Originally Posted by Jagboi
Thanks for the explanation! By single point of failure I meant that if Amadeus is down/unavailable/whatever, it appears that nothing moves at BA. Correct? If so, it means that everything is relying on that single data provider, so it's a "mission critical" link. Doesn't matter how good FLY is, if Amadeus is down nothing else matters. As a business, I wouldn't want everything to be relying on a single, outside provider to keep my business going, that seems like a large risk.
Are you suggesting that an airline should run two parallel reservations systems and two parallel inventory systems?
Globaliser is offline  
Old Oct 4, 2019, 2:26 am
  #30  
 
Join Date: May 2014
Posts: 7,237
Originally Posted by Jagboi
Thanks for the explanation! By single point of failure I meant that if Amadeus is down/unavailable/whatever, it appears that nothing moves at BA. Correct? If so, it means that everything is relying on that single data provider, so it's a "mission critical" link. Doesn't matter how good FLY is, if Amadeus is down nothing else matters. As a business, I wouldn't want everything to be relying on a single, outside provider to keep my business going, that seems like a large risk.

You've mentioned two failures/outages this year, that seems like a large number for something so important. I have friends in the telecommunications industry and I think the government mandated minimum reliability for landline phones is something like 99.9999% uptime, they called it "4 nines". I can't ever recall a time when a conventional landline was unavailable under normal circumstances ( i.e a hurricane hasn't just gone through the area). If I'm doing the calculation right, that's an outage of no more than 31 seconds per year. BA's systems don't quite seem to be at that level yet.
Sure, and the Airline industry works to that same level of reliability where it matters mostly. It's all about risk management. Engines, airframes, things that cause catastrophes and loss of lives if they fail (similarly to your telcom example): there, failure is not an option and redundancy - double, triple, often quadruple - is the must (which makes the MAX case all the more abnormal but I digress).

In the Amadeus' case, in the FLY case, the worst that can happen is that the airline reverts to manual procedures and flights get cancelled. An inconvenience, granted, but something that an airline is insured against and that can bounce back from, and BA has gotten better - it might not seem so, but there's no longer a pile of rotting bags on the tarmac as there was in T4 days.

When the worst that it could happen is another May Bank Holiday-like outage, or what happened to Delta a couple of years ago - so mass cancellations, millions wasted, Alex in a yellow tabard - and not death then it's a different matter. Especially because what you're advocating is the airline equivalent of having your car fitted with fully functioning wheels (steering, suspension, traction) on the roof so that, in the event it flips, you can still drive. If BA was to be fully hedged against Amadeus having a wobble they'd have to have a parallel system that can do exactly what Altea does - holding inventory, holding reservations, transferring flight schedules and reservations to the departure control systems at -72hr, weight and balance, check-in, boarding, lounges, conformance, doc checks, inter-airline-through-check - and that must be running in parallel, so that if I check flight BA1234 for Tuesday on 1A, and I check flight BA1234 for Tuesday on, say, Sabre, I get exactly the same data. And everyone must be able to flicker from one system to another in minutes.
NickB and Scots_Al like this.
13901 is offline  


Contact Us - Manage Preferences - Archive - Advertising - Cookie Policy - Privacy Statement - Terms of Service -

This site is owned, operated, and maintained by MH Sub I, LLC dba Internet Brands. Copyright © 2024 MH Sub I, LLC dba Internet Brands. All rights reserved. Designated trademarks are the property of their respective owners.