Dreamliner Battery Fire Solutions and Concerns

Don Tuite reviews the National Transportation Safety Board's report on the Li-Ion battery fire aboard a Boeing 787 Dreamliner at Logan Airport, Boston, and Boeing's proposed modifications to the battery and battery system.

I’ve been tracking Boeing’s and the aviation authorities’ responses to the events I originally wrote about in the blog titled; Dreamliners Shouldn’t Smoke. In the follow-ups, I’ve been helped considerably by a friend who is also an FAE at a company I report on, but he asks not to be identified, so I won’t, even though he’s the guy who came up with a key piece of information that I didn’t have and that explains the logic of Boeing’s proposal for a titanium battery box with stainless-tube vents to outside ambient.

Let’s go back to March 7, when the National Transportation Safety Board (NTSB) issued its initial report on the fire aboard the Dreamliner at Logan Field in Boston. This is a classis preliminary failure-analysis report. It contains narratives from eyewitnesses, especially first-responders, a description of the components of the electrical system, and damage photos. It’s all straight facts, nothing outrageously judgmental.

On January 7, 2013, at about 10 pm, the airplane arrived at Boston from Narita, Japan. After passengers and crew had left the plane, at about 10:21 pm, some people in the cleaning crew saw smoke in the aft cabin. About the same time, a maintenance manager in the cockpit noticed that the auxiliary power unit (APU).

APUs are small jet turbine/generator sets that provide the power to start up the main engines and provide cockpit and cabin power until the main engines take over. What’s important here is that the APU also charges the battery that starts it. There had been nothing unusual about the operation of the APU before or during the flight. After the plane landed in Boston, it had been turned on while the plane taxied to the gate. It was still on when all hell broke loose.

Right after the smoke and APU shutdown were observed, mechanic opened the aft electronic equipment bay (E/E bay) where the battery lived, “and found heavy smoke and fire coming from the front of the APU battery case.” He called the airport fire fighters and led one of them to the access door.

The firefighter saw “a white glow about the size of a softball” on his hand-held thermal imaging camera so he hit it with a shot of Halotron fire-extinguishing agent, which made the glowing image shrink. They hit it with more Halotron, set up a fan to try to suck smoke out of the compartment and tried, and finally succeeded, with some difficulty, to get the battery out of the plane. (It was partially blocked and parts of the “quick-disconnect” in the rack had melted.)

That’s the exciting part of the report, the part of with smoke and fire engines. What follows is a detailed description of the “post-mortem,” with explanations of how the components of the system work and photos of exactly what kinds of damage had occurred.

Since the battery is at the core of the problem, here’s the NTSB’s description of it and what it looked like after the event:

“The APU battery consists of eight lithium-ion cells that are connected in series and assembled in two rows of four cells. Each battery cell has a nominal voltage of 3.7 volts. The cells have a lithium cobalt oxide compound chemistry and contain a flammable electrolyte liquid.

“External observations of the battery involved in this incident showed, among other things, that the right side of the battery case appeared to have the most extensive damage of the four battery sides.3 Disassembly of the battery revealed that the cells that were located in the left side of the battery (cells 1 through 4) generally exhibited the least thermal and mechanical damage and that the cells that were located in the right side of the battery (cells 5 through 8) generally exhibited the most thermal and mechanical damage. Thermal damage was the most severe near cell 6. Continuity measurements using a digital voltmeter indicated that all of the cells were found to be electrically short circuited except for cell 8.

“The APU battery was configured so that each cell’s vent disc, which is a plate that ruptures when the internal pressure in a cell reaches a predetermined level, would be oriented toward the exterior of the battery. Disassembly of the battery showed that the vent discs on cells 1 through 3 were opened slightly, the cell 4 vent disc was intact (although weight measurements indicated that the cell lost some electrolyte), and the vent discs on cells 5 through 8 had opened more completely, leaving a ruptured appearance.”

Elsewhere in the report: “The APU battery installed on the incident airplane was manufactured by GS Yuasa Corporation in Kyoto, Japan, in September 2012 and was delivered new to Boeing. The battery had eight individual lithium-ion cells, all of which came from the same manufacturing lot produced by GS Yuasa in July 2012. The battery was installed in the incident airplane on October 15, 2012, and was initially charged by the airplane on or about October 19, 2012. Boeing records showed that the battery was disconnected (but not removed from the airplane) on December 5, 2012, as a precaution while an electrical power panel was inspected for foreign object damage. The battery was reconnected the next day.”

How is the battery managed? The report says: “Boeing contracted with Thales Avionics Electrical Systems of France to design the 787 electrical power conversion subsystem, which is part of the airplane’s electrical power system. Thales then subcontracted with various manufacturers for the main and APU battery system components. . . .”

A little further down: “The BCU [Battery Control Unit] for the APU battery was manufactured by Securaplane Technologies in June 2012. The BCU includes an electric connector for communication (among the BCU, battery, and airplane), a ground wire stud, and power terminals for the two large battery cables. Another BCU is used on the main battery installation.

“The BCU was examined at Securaplane’s facility in Tucson, Arizona, on January 22 and 23, 2013. The resistance of each pin in the J1 connector was measured to ground, and the results did not reveal any anomalies. Also, no anomalies were found during a visual inspection of the BCU internal components.

“The BCU was connected to test equipment, and an acceptance test procedure (ATP) was performed according to Thales and Securaplane reference documents. The BCU passed all performed portions of the ATP except for one test, which was designed to verify that the BCU would not send out a battery charging current if [the battery was too cold].”

(“Too cold” was around 5°;The BCU actually shut off current around 8°. No smoking gun there.)

I confess that I skimmed the report and didn’t get much out of it. My friend the FAE, who is skilled in finding out where his customers have screwed up seemingly bulletproof reference designs was more diligent. He said:

“It is pretty obvious that something happened to the inside of cell 5. It was the only cell that punched holes through the metal box.”

He further noted two critical quotes: “The inspections determined that arc damage occurred from contact between the cell 5 case and the battery case . . . . The other seven cells did not exhibit any evidence of electrical arcing on the exterior of the cells.’”

Then he goes on: “The case for cell 5 is located about 0.2 inch from the battery case The battery case exhibited no inward deformation at the protrusion, whereas the cell case exhibited outward expansion. These features are indicative of arc damage between the cases after expansion of cell 5.’”

He noted parenthetically, “(If cell 5 shorted internally the battery would expand from the heat)”

Then he dissected the time-line, noting, “ The bad event lasted 37 seconds. i.e.,

At 1021:01 the bus voltage drops one volt; three seconds later the battery is absorbing 44 amperes.

“Between 1021:7 and 1021:15 the battery voltage bounces around, then, from 1021:27 to 1021:30, the battery bus voltage decreases 1 V/s down to 28 V. At 1021:37, the battery voltage cycled to zero and back to 28 V three times with a draw of 4 A, after which the APU finally shut down.”

“This leads to his observation that cell 5 is in the middle of the battery array so it has ~16 V on one side and ~12 V on the other.

“Thus, if cell 5 shorted because lithium metal punctured the separator, which appears to be the case in some of the slides. The battery would short one cell and draw a lot of current. This could also be the short that made holes next to cell 5 because the battery case is at ground. The short would burn itself open and draw less current, but the cell would short again (maybe internal not to the metal case) progressively dropping the voltage down to 28 volts in a seven-cell battery.”

Fascinating stuff. It’s not possible to tell whether the NTSB engineers have made similar deductions, but there’s plenty to chew on that’s not covered in the general press. The next stage in our communications came on March 15, when the Bloomberg.com site published Alan Leven’s short report on contradictions between Boeing’s assertion that there was no fire on the aircraft and the NTSB report. In the story, Levin attributed a comment to Michael Sinnett, Boeing’s chief project engineer: “Boeing is redesigning its batteries to ensure a fire isn’t possible. Among the new features will be a fire-resistant stainless steel case that will prevent oxygen from reaching the cells so fire can’t erupt.”

That same day, a Chicago Tribune story by Gregory Karp said, “Boeing provided details Friday on how it will encase the redesigned power pack in a steel box, pack it with different insulation, heat-resistant material and spacers, drainage holes to remove moisture and to vent any gases from overheating directly to the atmosphere outside the aircraft.

“The fortified power pack can withstand 80 possible malfunctions covering all the potential failure scenarios that Boeing engineers could envisage, Sinnett said. ‘I would gladly have my family, my wife and my children, fly on this airplane,’ he said.

“The fortified power pack can withstand 80 possible malfunctions covering all the potential failure scenarios that Boeing engineers could envisage.”

That led to some more emails between my FAE friend and me. He waxed eloquent on the legendary creativity exhibited by Murphy in demonstrating the universality of his law, along with the added weight of the battery box and accessories negating the weight saving that led to the use of Lithium in the first place. I focused on the self-sustaining nature of Li battery fires.

Way back in August of 2006, after Dell and Apple recalled 12 million Li-Ion battery packs between them, I wrote a report in Electronic Design called The Fire Next Time, in which I referred to a study performed by the FAA office of aviation research: Flammability Assessment of Bulk- Packed, Nonrechargeable Lithium Primary Batteries in Transport Category Aircraft

The researchers tested batteries from a number of manufacturers by suspending individual batteries over fire pans charged with a quantity of 1-propanol in a 4-by-4-by-4-ft test chamber set up to provide the same Halon 1301 concentration used in a standard aircraft cargo compartment for initial fire knockdown.

“A relatively small fire source is sufficient to start a primary lithium battery fire. The outer plastic coating easily melts and fuses adjacent batteries together and then ignites, contributing to the fire intensity. This helps raise the battery temperature to the self-ignition temperature of lithium. Once the lithium in a single battery begins to burn, it releases enough energy to ignite adjacent batteries. This propagation continues until all batteries have been consumed.

“Halon 1301, the fire suppression agent installed in transport category aircraft, is ineffective in suppressing or extinguishing a primary lithium battery fire. Halon 1301 appears to chemically interact with the burning lithium and electrolyte, causing a color change in the molten lithium sparks, turning them a deep red instead of the normal white. This chemical interaction has no effect on battery fire duration or intensity.

“The air temperature in a cargo compartment that has had a fire suppressed by Halon 1301 can still be above the autoignition temperature of lithium. Because of this, batteries that were not involved in the initial fire can still ignite and propagate.

“The ignition of a primary lithium battery releases burning electrolyte and a molten lithium spray. The cargo liner material may be vulnerable to perforation by molten lithium, depending on its thickness. This can allow the Halon 1301 fire suppressant agent to leak out of the compartment, reducing the concentration within the cargo compartment and the effectiveness of the agent. Holes in the cargo liner may also allow flames to spread outside the compartment.

“The ignition of primary lithium batteries releases a pressure pulse that can raise the air pressure within the cargo compartment. The ignition of only a few batteries was sufficient to increase the air pressure by more than 1 psi in an airtight 10-meter-cubed pressure vessel. Cargo compartments are only designed to withstand approximately a 1-psi pressure differential. The ignition of a bulk-packed lithium battery shipment may compromise the integrity of the compartment by activating the pressure relief panels. This has the same effect as perforations in the cargo liner, allowing the Halon 1301 fire suppressant to leak out, reducing its effectiveness.”

That is what was in the back of my mind when I wrote to my friend about my self-starting-self sustaining concerns. He directed me to a Wired piece by Jason Paur, Boeing Says Dreamliner Battery Redesign Eliminates Chance of Fire.

Paur wrote, “Boeing plans to more thoroughly protect the eight individual cells within the 63-pound battery. The battery itself will be installed inside a stainless steel box with walls one-eighth of an inch thick.” The box, he says, is not simply designed to contain a fire, but to prevent one from starting.

According to the article, Mr. Sinnett said, “This enclosure keeps us from ever having a fire to begin with,” he said, referring to the newly designed container. He said there is not a sufficient amount of oxygen to “contribute to combustion” should the battery overheat or experience a short circuit. During six weeks of testing, he said. Engineers used external heaters to intentionally overheat the battery in an attempt to induce a short. When the cells subsequently vented hot electrolyte as designed, engineers tried without success to ignite the vented gasses. Even after pumping additional oxygen into the box failed to ignite the mixture for more than an instant.

“There was a small amount of combustion for 200 milliseconds and it went out again.”

Paur wrote, “Should a cell rupture within the redesigned steel box, the gases would open a pressure release disc, a valve of sorts, which would vent the electrolyte mixture through a one-inch titanium tube to the exterior of the aircraft. Each of the two batteries has its own vent tube, and the system is designed to eliminate the chance of gasses or smoke emanating from a failed battery entering the aircraft. Boeing says the vents will require placing two small holes in the fuselage, but they will not affect the structure of the aircraft. The FAA requires that the holes do not result in the vented gases reentering the aircraft downstream.”

My friend turned up another source, a British site called The Electropaedia. It’s a service of Woodbank Communications Ltd, a battery consultancy.

This diagram is the skeleton key to the self-sustaining solution. (See the Figure).

What it says is that while the electrolyte is flammable and releases flammable gases through the vent in each cell, if the temperature rises to approximately 100° C, The battery doesn’t make its own oxygen until about 270° C. (Note that these guys are not endorsing Boeing; they simply have a Website with a graphic that explains what the engineers at Boeing are probably thinking.)

When I was a young engineer, working at Garrett AiResearch, I had a chat in the break room with the guy who ran all of our C5A subprograms, about the Boeing 747, which was still several years from its first commercial flight. He cautioned me to avoid flying in a new commercial airplane until it had been in service a few years. Not because it was particularly likely to fall out of the sky, but because it was very likely to exhibit bugs that would range from passenger discomfort to flight delays and cancellations. I’ve always found that to be good advice.

The problem I have with this approach, if I understand the report correctly, is that Boeing has not fixed the underlying problem that caused the short and fire. They have designed a containment and venting system they believe will contain any malfunction and prevent any spread beyond the battery box. This is not altogether comforting.

The danger of this failure mode has been ameliorated. Even if the batteries do manage to catastrophically fail, there is no danger to the passengers.

Still for me, I'm following the "avoid flying in a new commercial airplane until it had been in service a few years."

In much the same way I won't touch a Microsoft product until the second service pack.

Halon 1301, the fire suppression agent installed in transport category aircraft, is ineffective in suppressing or extinguishing a primary lithium battery fire.

This is not altogether comforting either, contained or not.

I know that in other contexts you basically have to let a lithium fire burn itself out, and that is what's happening here by design.

"I have pondered both of your replies and can’t decide which one weighs more heavily on my mind."

Just picture flying above a small bomb which has a containment vessel built surrounding it to limit any explosion. Sleep tight. Also, as null and void stated, Boeing has lost the reason, or at least part of it, for using this type of battery which was less weight and smaller size.

Boeing engineering management seems hell-bent on keeping the Li-Ion battery no matter what. I think that prudence would dictate using a battery technology that had a more extensive track record. But that will cost weight and volume and a small price to pay for not losing an aircraft.

They went with the lightest weight chemistry available: lithium cobalt oxide.

They are now boxed into a corner, the entire electronics support package is optimized to that particular lithium technology.

If they revert to NiCad or even a different lithium chemistry they need to redesign all the support circuitry.

Then they need to spend a couple years torturing the batteries and support circuity to make sure they haven't overlooked a new and improved failure mode!

The it gets really bad, they need to make absolutely sure that no one in any third world cesspit repair facility mates the new batteries to the old electronics or vice-versa. Ever, not even once.

I remember reading a Theseus about creating energy for virtually nothing.

It is a known fact that any liquid or gas passing between two parallel plates creates power.

One could put multiple layers on ships, airplanes, cars, etc and have power left over to power other devices

Why don’t they just go back to the older batteries? Or was there a problem with them?

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

The way I read this, it was a purely mechanical patch. To change battery chemistry (go back to the old batteries) would require electrical system redesign and reverification. That would be way more time, money, and labor intensive than the pure mechanical band-aid.

It is a known fact that any liquid or gas passing between two parallel plates creates power.

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Please elaborate!

Why don’t they just go back to the older batteries? Or was there a problem with them?

Probably the machinery that this battery powers is too power-hungry. Safe Nickel/Cadmium/Silver chemistries are far less capable, and there is not enough space to install several of those batteries. (There is barely enough space for one Li-ion battery.) The airplane is heavily electric - it is not cable-operated, as they used to do in 1920s, and it is not hydraulic, as many airplanes are still built today. When you need electric power you cannot pick between operating the left wing vs. the right wing - you need both, otherwise the airplane will flip.

I agree with the opinion that the cause of the fire remains unknown. Most likely it is a singular manufacturing defect. Those batteries are not built as accurately as an integrated circuit; they are just plastic bags with metal foil and assorted chemicals, roughly rolled together. There is no way to assure repeatability; and QA testing is not likely to discover a short that hasn't happened yet.

However if the failure of the battery is no longer a danger then what can I say? Airplanes are not something that I use these days. (I'm still waiting for TSA to be disbanded.)

Thank you for your replies.

I have an old Dodge pickup. It is made of metal. I can open the hood and fix most things. There is compartment space to do that too.

My wife has a “new” car. It is made of plastic. I opened the hood once. Now I just call Bob. And he calls Steve, and Steve calls...and so on...and so on...

They both get about the same gas mileage.

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.