Posted on 04/02/2020 9:41:53 AM PDT by dayglored
US air safety bods call it 'potentially catastrophic' if reboot directive not implemented
The US Federal Aviation Administration has ordered Boeing 787 operators to switch their aircraft off and on every 51 days to prevent what it called "several potentially catastrophic failure scenarios" including the crashing of onboard network switches.
The airworthiness directive, due to be enforced from later this month, orders airlines to power-cycle their B787s before the aircraft reaches the specified days of continuous power-on operation.
The power cycling is needed to prevent stale data from populating the aircraft's systems, a problem that has occurred on different 787 systems in the past.
According to the directive itself, if the aircraft is powered on for more than 51 days this can lead to "display of misleading data" to the pilots, with that data including airspeed, attitude, altitude and engine operating indications. On top of all that, the stall warning horn and overspeed horn also stop working.
This alarming-sounding situation comes about because, for reasons the directive did not go into, the 787's common core system (CCS) an Intel Wind River VxWorks realtime OS product, at heart stops filtering out stale data from key flight control displays. That stale data-monitoring function going down in turn "could lead to undetected or unannunciated loss of common data network (CDN) message age validation, combined with a CDN switch failure".
Solving the problem is simple: power the aircraft down completely before reaching 51 days. It is usual for commercial airliners to spend weeks or more continuously powered on as crews change at airports, or ground power is plugged in overnight while cleaners and maintainers do their thing.
The CDN is a Boeing avionics term for the 787's internal Ethernet-based network. It is built to a slightly more stringent aviation-specific standard than common-or-garden Ethernet, that standard being called ARINC 664. More about ARINC 664 can be read here.
Airline pilots were sanguine about the implications of the failures when El Reg asked a handful about the directive. One told us: "Loss of airspeed data combined with engine instrument malfunctions isn't unheard of," adding that there wasn't really enough information in the doc to decide whether or not the described failure would be truly catastrophic. Besides, he said, the backup speed and attitude instruments are for obvious reasons completely separate from the main displays.
Another mused that loss of engine indications would make it harder to adopt the fallback drill of setting a known pitch and engine power* setting that guarantees safe straight-and-level flight while the pilots consult checklists and manuals to find a fix.
A third commented, tongue firmly in cheek: "Anything like that with the aircraft is unhealthy!"
A previous software bug forced airlines to power down their 787s every 248 days for fear that electrical generators could shut down in flight.
Airbus suffers from similar issues with its A350, with a relatively recent but since-patched bug forcing power cycles every 149 hours.
Persistent or unfiltered stale data is a known 787 problem. In 2014 a Japan Airlines 787 caught fire because of the (entirely separate, and since fixed) lithium-ion battery problem. Investigators realised the black boxes had been recording false information, hampering their task, because they were falsely accepting stale old data as up-to-the-second real inputs.
More seriously, another 787 stale data problem in years gone by saw superseded backup flight plans persisting in standby navigation computers, and activating occasionally. Activation caused the autopilot to wrongly decide it was halfway through flying a previous journey and manoeuvre to regain the "correct" flight path. Another symptom was for the flight management system to simply go blank and freeze, triggered by selection of a standard arrival path (STAR) with exactly 14 waypoints such as the BIMPA 4U approach to Poland's rather busy Warsaw Airport. The Polish air safety regulator published this mildly alarming finding in 2016 [2-page PDF, in Polish].
This was fixed through a software update, as the US Federal Aviation Administration reiterated last year. In addition, Warsaw's BIMPA 4U approach has since been superseded.
The Register asked Boeing to comment. ®
It wouldn’t have been to the open internet but to the company’s intra-net.
PS: it’s up to maintenance to ensure of the upgrades. When we fire up the system it had better be the latest version; this is Airbus. I don’t know how Boeing does it.
“Ok it is Linux.”
So, you only heard of Linux so everything is Linux?? VxWorks is NOT linux.
> PS: its up to maintenance to ensure of the upgrades. When we fire up the system it had better be the latest version; this is Airbus. I dont know how Boeing does it.
Ah, ok. IMHO whatever it is, it needs to be secure with no access from the general internet, and there needs to be an orderly system in place so that others such as pilots and mechanics can work with some assurance about what they are working with.
That's a really good point. I'm sure it's costly in terms of time, at least.
Ok, thanks anyway, and I appreciate your thoughtful answer.
Agreed. Have to submit a change, maybe even go to CAB... bleh just to upgrade Cisco ios on a couple stacks.
You know why? It saves a couple of pennies per car not to use any flash memory for it. Same reason they dont put spare fuses or fuse puller in cars anymore. Every design goes through reviews for cost not benefit.
I hated working automotive but it did teach me lots about cost trades.
Learn something new every day!
The windows ARE pretty cool on a 787. I only hope they don’t have to open and close them on a recycling period that lines up when I’m sitting next to them at 38,000 feet
I don’t have auto-updates turned on. I would imagine any updates are done manually, with someone updating the firmware on board.
Software updates are performed on Boeing and Airbus aircraft frequently. Navigation databases are updated to the flight control computers every 30 days. This is usually accomplished with a portable computer taken on board the aircraft and connected to a data port in the cockpit. 787’s and A350’s have the ability to receive software over wifi. That data is then downloaded by a mechanic from the inbox or repository to the destination hardware. This is done in the flight deck. A mechanic has to verify that the software is the latest and then install it to the particular destination. It cannot be accomplished remotely. Boeing isn’t the only aircraft that require power to be cycled on and off routinely. Airbus 330’s and 350’s do also.
My mistake, nav databases are loaded to the flight management computers, not flight control computers. As for cycling aircraft power, 20 minutes off before restarting is typical.
> Software updates are performed on Boeing and Airbus aircraft frequently. Navigation databases are updated to the flight control computers every 30 days. This is usually accomplished with a portable computer taken on board the aircraft and connected to a data port in the cockpit. 787s and A350s have the ability to receive software over wifi. That data is then downloaded by a mechanic from the inbox or repository to the destination hardware. This is done in the flight deck. A mechanic has to verify that the software is the latest and then install it to the particular destination. It cannot be accomplished remotely.
I did not want to go into details because I was not certain, but thanks for filling in those details. I would imagine that what you described is very secure.
How many pilots keep their 787 idling 51days?
Its a big deal to completely shut down a modern aircraft and then power it all back up. And economically speaking, the airline wants that plane in the air carrying passengers as much as possible. So its not idling any more than necessary, and its not getting power cycled unless absolutely necessary.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.