Feds on Waco: Shooting
in the dark
Posted: April 12, 2003
1:00 a.m. Eastern
Editor's note: Barbara Grant is an electro-optical engineer specializing in the measurement of light. She has studied extensively the issues surrounding the federal government's probe into the Waco siege and recently completed this comprehensive research paper on the subject.
By Barbara Grant
© 2003 Barbara Grant
Science offers an unbiased approach to problem solving, but good scientific practice can be jeopardized when controversial topics are at issue. At Waco, government science failed. To illustrate, compare the problem-solving on Waco technical issues with NASA's approach to getting a crippled Apollo 13 spacecraft back to Earth. Who can forget the response to "Houston, we have a problem?" Swiftly, decisively, in a zero-tolerance-for-error atmosphere, engineers, scientists, flight controllers and astronauts formulated a solution that brought the spacecraft home. Calculations were made and checked, models built, hypotheses generated and tested. Many individuals applied their expertise, and the work was folded into a solution. NASA's "successful failure" is something Americans are proud of; it demonstrates how well ingenuity and know-how can operate when facing a critical technical challenge.
Having demonstrated this ability on Apollo 13, at Waco, we forgot we possessed it. Waco's critical challenge was to determine if Federal agents1 fired on members of the Branch Davidian religious group. Unfortunately, government investigations treated this problem not as a scientific issue, but as a legal matter in which they would play the roles of judge and jury. The following paragraphs amplify this claim.
The problem is simply stated. On April 19, 1993, a Forward Looking Infrared (FLIR) imaging sensor circled Mt. Carmel onboard an FBI surveillance aircraft. The sensor recorded many bright flashes before the complex burned to the ground. The key issue was whether flashes recorded on infrared imagery represented the signatures of weapons fire or some other phenomenon, such as solar reflections off debris. Non-government experts concluded that the flashes were gunfire; experts retained by the government concluded they were not. Three experts presented reports for government investigations, one for the U. S. House of Representatives' Committee on Government Reform and two for the Office of Special Counsel headed by former U. S. Sen. John Danforth. The FLIR controversy received considerable attention in 1999-2000 and was one of five issues raised by Davidian survivors and relatives in a wrongful death lawsuit against government officials.2
There are problems with the government's scientific approach in reaching the "we did not fire" conclusion, and this paper will discuss them. It will show that dogmatic conclusions by government experts are no substitute for the application of real science and engineering as evidenced by NASA on Apollo 13. The paper describes the expertise needed to fully resolve the Waco FLIR controversy and contrasts that with the expertise actually demonstrated in government studies. It also critiques testing performed under the auspices of the Special Counsel. As will be seen, the government's solution strategy fell far short of good scientific practice.
Analysis by experts, judgment by decree
Each side in a trial endeavors to win its case. To this end, each side commissions experts who've arrived at conclusions favorable to the cases of those who've retained them. Experts from opposing sides in litigation do not sit together over coffee and discuss the technical merits of their cases in the interest of science; and one would not expect experts retained by plaintiffs and defendants in the Davidians' wrongful death lawsuit to work out scientific details among themselves.
An analysis also exhibits the character of a solo event, as the analyst provides his or her best interpretation of the data under study. An investigation, however, is different, particularly when it presents itself as the vehicle for uncovering the truth on a disputed matter, as the Special Counsel's investigation into Waco did. While analysis results fold into an investigation, they should be weighed at a higher level. In particular, a distinction between persons and issues should be made. Rather than assuming that the conclusions of its analysts were correct (apparently because they were retained by the Special Counsel) the investigation should have adopted an issues-oriented perspective, noting discrepancies among many analysts on key issues and working to resolve them.
Given the differing interpretations of the Waco data existing at the time of the Special Counsel's inquiry, a good first step toward problem resolution would have been to draw together analysts holding contradictory views, allowing them to go through their arguments point by point. Additional personnel could have been sought and retained to provide peer review – the detailed critique of assumptions, methods, and conclusions that is essential to scientific inquiry. At a minimum, the process would have ensured that technologists who'd worked the Waco flash problem were given the opportunity to critique the analyses of their peers before government judgments decided the issue. This did not happen. Special Counsel experts Vector Data Systems Ltd. of the UK and Lena Klasen and Sten Madsen of Sweden concluded that solar reflections caused most Waco flashes. The Special Counsel adopted this conclusion. Thus, the operational definition of "truth" in the investigation was the concurrence of two expert opinions.
The conclusion of the House of Representatives' Committee on Government Reform was even less robust. Physicist Don Frankel of Photon Research Associates found that the FLIR tapes did not contain evidence of gunfire. This conclusion was opposite to that of infrared analyst Carlos Ghigliotti, who had also been retained by the House Committee.3 Ghigliotti, however, suffered a fatal heart attack before delivering his final report. Whether or not the committee intended that the analysts meet and discuss their arguments is unknown. As it stands, the committee's verdict suggests that "truth" is the judgment rendered by the living analyst.
While the House committee's inquiry did not have the resources that the Danforth commission possessed, both approaches suffered the same flaw. Emphasizing persons rather than issues in an investigation serves only to propagate controversy.
The issues-oriented perspective is best illustrated by an example. Addressing the topic of the Waco FLIR imager manufactured by British firm GEC-Marconi, Frankel noted:
"The FLIR video technology has a very low probability of detecting small-arms muzzle flash." 4
To support his claim, Frankel reviewed camera characteristics and compared them to muzzle flash literature and the test results of colleagues. Many months earlier, "60 Minutes II" aired a segment in which Jane's Information Group analyst and spokesman Paul Beaver concluded that the Waco flashes were gunfire. Commenting on the ability of the camera to detect gunfire signatures, he told The Dallas Morning News:
"You're looking for just that. … I have personally been in a situation where I've seen gunfire, using the GEC-Marconi system. … In a firefight situation, it's very, very useful to detect where the enemy is." 5
Either the Waco FLIR could be used effectively for imaging muzzle flash, or it was essentially useless for that purpose; both arguments cannot be right. Failure to look at the problem on the basis of issues such as these paves the way for continued dispute and is a major flaw in the government's investigative strategy.
Beaver's comments point to another flaw in Waco investigations: the lack of appropriate expertise brought to bear on the problem.
Gunfire issues, scientific experts
Critical to the success of the Apollo 13 mission was the diversity of background applied to it. While some engineers may believe that they can push buttons inside a flight simulator as well as any astronaut can, an individual training with the technology daily will have mastered its nuances and be able to execute maneuvers in the same manner as those in space whose lives depend on them.
Similarly, technologists trained in the scientific side of FLIR imaging – which includes disciplines such as infrared instrumentation, imagery interpretation, video engineering and atmospheric analysis – may not possess a background in muzzle-flash phenomenology, weapons or tactics. As allegations of gunfire generated the controversy, disciplines bearing upon gunfire must be key to its solution. The paucity of such expertise in official inquiries is testament to the government's failure to investigate well.
For example: Individuals such as Beaver who'd seen gunfire on a similar FLIR were not tasked to determine whether gunfire generated the flashes on the Waco FLIR. Individuals with combat experience did not contribute tactical insights to the analyses. Weapons experts familiar with the many variables that can influence muzzle flash were not among those whose reports dismissed this phenomenon. Technologists who spend significant portions of their careers studying muzzle flash were absent from the analyst roster.
Instead, the final judgment relied on the conclusions of imagery experts such as Vector Data Systems. Addressing the contentious issue of flashes emerging from dark objects6 behind a combat engineering vehicle (CEV) advancing on the Mt. Carmel gymnasium, Vector noted that its imagery analysis "refutes the theory that a person would lie or crouch in such proximity to the very hot CEV engine." 7 The analysts also pointed out that the vehicle rolls over the shapes as it backs away from the gym.
Looking at the problem tactically provides a different perspective. If the "dark objects" were personnel expecting fire from Davidians inside the gym, lying or crouching behind the vehicle would be preferable to being shot. 8 If the vehicle was equipped with a hatch allowing personnel to exit beneath the tank, individuals might lie beneath it without being crushed. The Vector analysts' conclusion that these flashes were generated by heat reflections off debris appears to be based not only on imagery, but also on behaviors they judge to be improbable. A more well-rounded approach would involve assuming that such behaviors are probable; assessing the amount of spatial detail that the camera might record at the altitude at which images were obtained; and determining whether the quality of the videotape used for analysis was sufficient to exploit camera capabilities.
Dismissing gunfire as a cause of the flashes appeared to be relatively easy for some. Imagery experts Klasen and Madsen, whose analysis focused primarily on solar reflections, noted that the work of the Maryland Advanced Development Laboratory (MADL), an expert witness for the Justice Department in the civil lawsuit, provided "a much more accurate and correct explanation of the flashes seen on the April 19, 1993, FLIR imagery"9 than the work of analysts who'd concluded gunfire had been their source.
One wonders how descriptors such as "accurate" and "correct" can be applied by authors who have not studied gunfire. What Klasen and Madsen failed to note was that MADL's conclusion – that most flashes on the Waco tape lasted too long to have been generated by gunfire from weapons thought to have been in use at Waco – was based on extrapolating test results from single-shot firings. 10 Given that the phenomenology of single-shot firing is different from that of multiple firings, 11 the approach is open to question. Lack of background in the discipline guarantees that the question will not be asked.
The lack of weapons expertise has other implications. Might analysts know that factors such as barrel length, ammunition type and flash suppressants influence muzzle flash signatures? Is a literature search sufficient to gain understanding of these variables, or must testing be performed? And if the latter, in what kinds of conditions must tests be conducted to ensure valid comparison with the conditions at Mt. Carmel?10
Weapons, debris and data anlysis
"Re-creating" the conditions on April 19, 1993, in order to "simulate" events on the standoff's last day is an extremely difficult proposition. Many factors go into the mix – weather conditions, soil characteristics; amount, type, location and orientation of reflective material; number and type of weapons alleged to have been carried by federal agents; vehicles, aircraft and FLIR sensor – among others. Like a complex choreography comprising difficult dance steps and the transitions that weave them together, a Waco "simulation" would have to account for these factors in the manner in which they were thought to have contributed to events. As both "steps" and "transitions" at Waco are matters of dispute, a "re-enactment" is very difficult to achieve.
The FLIR trial, conducted at Ft. Hood, Texas, in March of 2000 under the auspices of the Special Counsel, was not a re-creation, simulation or re-enactment of April 19 events. Rather, it was an effort to gather data that might be used in analyzing the Waco FLIR tapes. While a detailed critique of the effort is beyond the scope of this paper, examination of some of its aspects can prove useful. Two subjects are of particular interest: solar reflections and muzzle flash.
The Ft. Hood test proved that solar reflections can be imaged with FLIR. At the time of the test, this was a contentious issue. Infrared engineer Edward Allard, the plaintiffs' expert in the wrongful death lawsuit, maintained that solar reflections from debris could not have caused the Waco FLIR flashes due to the inability of the Waco instrument to image them.13 The test proved that contention to be inaccurate, inasmuch as solar reflections can be detected with FLIR. How Allard's analysis of the flashes might have evolved had he interpreted Ft. Hood data is unknown; unfortunately, he suffered a stroke prior to the test.
The test also showed that multiple, repetitive flashes of elongated shape can be generated by debris reflections. This, too, was an important point, given the repetitive nature of some of the Mt. Carmel flash events and their elongated shapes.
However, the conditions under which reflections were imaged at Ft. Hood did not duplicate those at Mt. Carmel. The Ft. Hood debris field consisted of five eight-foot squares, a large area of potentially reflective targets. Reflective debris at Mt. Carmel were scattered due to demolition of the complex, falling to the ground among building material and dirt. Ft. Hood debris had been watered down and covered with tarps prior to the test. Although this created a good field condition for observing the occurrence of reflections, the density of debris material makes correlation of specific flashes with the pieces that generated them very difficult to accomplish.14 In short, while the Ft. Hood test produced solar reflections, it did not do so under conditions approximating those at Waco.
Gunfire testing at Ft. Hood exhibited a similar shortcoming. Data from Ft. Hood firings produced infrared muzzle flashes lasting for shorter durations than the flashes on the Waco tape. Yet the firing area had been watered down prior to the trial15 and the atmospheric particulate generated at Mt. Carmel by the effects of wind and structural demolition was not accounted for in testing. Subsequent testing has shown that firing a weapon through particulate can enhance the flash's infrared duration; 16 survey of the literature on muzzle flash reveals that firing a weapon near dry ground will cause dust to be lifted aloft.17 Adding moisture to the soil prior to a firing test works to diminish this effect and is not representative of Mt. Carmel conditions after demolition began.
Another factor bearing upon the test's validity deserves comment. Months after the Special Counsel issued its judgment, investigator and filmmaker Michael McNulty determined, from a review of video and photographic evidence, that a weapon and ammunition combination present at Waco was not tested at Ft. Hood.18 As the combination of a short-barreled weapon and commercial-grade ammunition had been seen to produce bright, long-duration flashes,19 the discovery was significant. McNulty's findings were incorporated into a film "The FLIR Project," and a report sent to members of Congress.20
Most analysts who concluded that federal agents did not fire at Waco based their gunfire duration criteria on material other than the muzzle flash test results at Ft. Hood. 21 The exception to this generalization was Vector Data Systems, whose report made extensive use of FLIR trial data.22 Thus, results of a test conducted under very different conditions from those at Waco contributed significantly to the Special Counsel's final judgment.
Scientific fallout
Having considered several complications in the search for "the truth" of what happened at Waco on April 19, 1993, it is instructive to consider what "the truth" tells us about some of the science behind the flash controversy. This consideration will be pursued, as above, with respect to both muzzle flash and solar reflections.
Fact Number One: The infrared signature of small arms fire does not last long enough to have generated the flash signatures at Waco.
Although analysts whose reports contributed to the final judgment placed varying upper limits on muzzle flash duration, all three agreed that it does not last long enough to have generated the flashes on the Waco FLIR tape.
Therefore, data to the contrary showing muzzle flashes lasting long enough to have generated the flash signatures at Waco can be assumed to be (1) a hoax; (2) a misunderstanding; or (3) new science. If (3), continuation of research such as that described in Zegel and the work of McNulty should be considered for a National Science Foundation grant, as it can serve to advance basic understanding of a physical phenomenon.23
There is another possibility. One can hypothesize that data to the contrary could arise due to the characteristics of the particular FLIR used to record it. The Klasen-Madsen authors concluded that just such a characteristic was present within the Waco FLIR imager – that "thermal energy of short duration could appear longer in duration than in real life"24 due to sensor effects.
It's a curious position, given that the authors also adopted the view that muzzle flash is of insufficient duration to have generated the Waco flashes. Could such a camera characteristic have allowed muzzle flashes at Waco to last longer than they normally would? This possibility was not addressed in the report; the Special Counsel's judgment means it can't happen, anyway.
Fact Number Two: Solar reflections from debris emerged in appropriate geometrical position to reach the aircraft's FLIR sensor at the times most flash events were recorded.
This fact is critical. Even if muzzle flash and solar reflections appear "identical" in shape, size, brightness and duration, a solar reflection will not be recorded by the FLIR if the aircraft carrying it is not in appropriate alignment with respect to the sun and the debris.
Two opposite positions on this issue were presented. First was that of mathematician and imagery analyst Maurice Cox,25 who'd concluded that the aircraft was not in position to record solar reflections during the times that six key flash events occurred. In an effort to make the flash data fit a solar reflection hypothesis, Cox postulated several scenarios in which reflective material fell onto uneven ground or were "aimed" at the aircraft sensor by conceptual "machines" before concluding that such "aiming" was unlikely to occur naturally.26
On the other hand, Klasen and Madsen determined that proper geometrical alignment between sun, debris, and aircraft did exist during most flash events. Their conclusion was based on a detailed computer model that compared the appearance of flashes with the sun's known position and the aircraft's reconstructed path around the complex. The authors noted that their modeling technique produced aircraft positions that were "accurate and reliable";27 where they located the aircraft during flash events is not mentioned in the report.
If ever a discrepancy within the Waco FLIR problem cried out for the "issues-oriented" approach described earlier in this paper, this was it. Not only did this critical issue not receive such treatment; the Klasen-Madsen document did not appear on the internet until the Special Counsel's final judgment was issued, precluding public comment before the entire matter was deemed "closed."
The Klasen-Madsen authors reviewed Cox's work in their report, but unfortunately, misinterpreted its arguments.28 Cox wrote to the Special Counsel in November of 2000, and has not yet received a reply.
Conclusion
It would be difficult to imagine investigations beset with as many inadequacies as those purporting to offer the last word on the Waco FLIR flashes; nevertheless, official conclusions removed a controversial issue from the national stage. Skeptical consideration of investigative results is justified, particularly in view of the immense resources available to the government.
While the analyses of experts can form a good starting point for investigation, authorities erred by automatically adopting the conclusions of their experts as "the truth." Expertise in the critical areas of muzzle-flash phenomenology, weapons and tactics was missing among analysts whose reports decided the issue. A test still referred to by some as a "re-enactment" of events on April 19, 1993, failed to duplicate (or approximate) conditions at Waco; yet its results contributed significantly to the final judgment. The judgment, exonerating federal agents from wrongdoing at Waco, is predicated on "facts" about muzzle flash and solar reflection conditions at Mt. Carmel that, at the very minimum, deserve considerable additional scrutiny.
Investigative flaws have been noted and described. However, had imagery analysis produced definitive evidence of shooters at Mt. Carmel, the need for a problem-solving approach encompassing a breadth of expertise and tackling a variety of issues would go out the window. This, too, is a contested matter. Special Counsel expert Klasen, experienced in a wide variety of computer image processing techniques, found no evidence of human motion on the Waco tapes in proximity to the flashes.29 Vector's imagery analysts concluded likewise. Veteran imagery analyst Carroll Lucas, however, determined that personnel could be seen within the Mt. Carmel complex on several occasions prior to the fire.30 Former Air Force imagery analyst Michael Weatherford later determined that human figures appeared on the video as "soft fuzzy blotches," several in proximity to flashes, whose presence he'd detected by their "blocking out the rough background they are crossing."31
Missing from the argument was the final report of analyst Carlos Ghigliotti. Unwilling to base his imagery study on data other than the best available, he'd demanded and received a first-generation analog copy of the FBI's FLIR video, whose production from the original he witnessed.32 His methodology involved using both FLIR imagery and visible light video to gain an understanding of events;33 and he'd claimed to have correlated a FLIR flash with a visible image of a shooter.34 After his death, a staffer from the House Committee removed all material deemed "Committee property" from Ghigliotti's lab.35
A final factor deserves mention. While individuals of several technical backgrounds are capable of addressing the Waco problem, all are doubtless aware that the FLIR flash controversy does not exist within a scientific vacuum. This author noted that technologists unconnected to the controversy did not wish to have their names associated with the strictly technical observation that solar reflections can be imaged with infrared equipment.36 Attorney and investigator David Hardy has written of infrared experts who'd concluded that the FLIR flashes represented gunfire, but did not wish to become involved in a trial.37
Perhaps the most telling comment comes from infrared analysis firm Infraspection Institute, who'd analyzed the FLIR video for "60 Minutes." While noting that the firm's expert had concluded that the flashes were gunfire, "due to the potentially sensitive nature of the material, and the potential negative repercussions to Infraspection, we are choosing to decline any further comment."38
Such statements presage a sad day for science; sadder still, for the truth that science may uncover.
End notes:
1. This paper uses the term "federal agents" to refer to individuals acting on behalf of the Federal government The author does not assume a specific agency affiliation on the part of such individuals.
2. Federal Judge Walter Smith separated the FLIR issue from the other four, with the result that the FLIR was not discussed at trial.
3. More specifically, Ghigliotti claimed that FBI agents fired their weapons at Waco on April 19, 1993, David T. Hardy and Rex Kimball, "This is Not an Assault", Xlibris, 2001, p. 113.
4. Donald Frankel, "Assessment of Waco, Texas FLIR videotape," (Report to U. S. House of Representatives' Committee on Government Reform) Sept. 11, 2000, p 1. Frankel's arguments may also be seen in a paper of the same title, "Proceedings of SPIE," the International Society for Optical Engineering, 4370-51, 2001, pp. 286-300.
5. Lee Hancock, "FBI cameras at Waco same as ones used by British military, expert says," The Dallas Morning News, Jan. 29, 2000.
6. Many who claim that the government fired at Waco allege that these "dark objects" are men; and that the flashes emerging from these positions are weapons fire.
7. Vector Data Systems, Ltd., "Imagery Analysis Report: The Events at Waco, Texas, 19 April 1993," (Report to Office of Special Counsel) May 5, 2000, p. 46.
8. SFC Steven M. Barry, USA (Ret.) communication to author, 2000.
9. Lena Klasen and Sten Madsen, "Waco Analysis: Image Investigation and Video Authentication," (Report to Office of Special Counsel) October 4, 2000, p. 84. Some of the arguments in the report to the Special Counsel may also be found in Klasen, "Waco Investigation: Analysis of FLIR Videotapes," Proc. SPIE 4370-50, 2001, pp. 271-285.
10. Maryland Advanced Development Laboratory, "Analysis of Flashes Recorded by an Infrared Sensor," Feb. 28, 2000, p. 6.
11. See, for example, Gunter Klingenberg and Joseph Heimerl, "Gun Muzzle Blast and Flash," American Institute of Aeronautics and Astronautics, 1992, p. 203, for a discussion of the effect of firing a burst of rounds. In brief, such firings may generate a secondary combustion effect, causing muzzle flash to appear brighter and last longer than the flash due to a single shot.
12. See, for example, U. S. Army Materiel Command, "Engineering Design Handbook. Guns Series, Muzzle Devices," AMCP 706-251, Washington, 1968, and "Engineering Design Handbook, Elements of Armament Engineering, Part One, Sources of Energy," AMCP 706-106, Washington, 1964, for a discussion of some of the variables influencing muzzle flash.
13. Edward Allard, "Analysis of the April 19, 1993 Waco FLIR Videotapes," (Report to Davidian attorneys in civil lawsuit) March 1, 2000, p. 2-13.
14. Further detail on this subject can be seen at Maurice Cox, "Open Letter to Special Counsel Danforth,"
http://www.rolandresearch.com, Nov. 20, 2000.
15. A factor allowed by the test protocol. Vector Data Systems, "Protocol for a Forward-Looking Infra-Red Imagery Trial," February 16, 2000, p. B-1.
16. Ferdinand H. Zegel, "Infrared Signatures of Small Arms Weapons Fire," Proc. SPIE 4370-52, 2001, p. 311.
17. U. S. Army Materiel Command, AMCP 706-251, op. cit., pp. 2-3 - 2-4.
18. Jon Dougherty, "FBI weapon not tested in Waco probe," WorldNetDaily,
http://www.worldnetdaily.com, May 12, 2001.
19. Zegel, op. cit. p. 310.
20. Jon Dougherty, "Senators take fresh look at Waco evidence," WorldNetDaily, Jan. 17, 2002.
21. Duration contentions of several analysts are listed in Barbara Grant and David Hardy, "Muzzle flash issues related to the Waco FLIR Analysis," Proc. SPIE 4370-53, 2001, p. 317.
22. Vector Data Systems, op. cit
23. Unfortunately, Ferdinand Zegel is not around to continue the work, having passed away in 2002. He will be missed.
24. Lena Klasen and S. Madsen, op. cit., pp. 27-28.
25. Maurice Cox, "Sun Reflection Geometry Technical Report and Appendix," at
http://www.rolandresearch.com, Nov. 20, 1998.
26. This is particularly important with respect to the case of multiple flash events, in which several reflections must reach the aircraft sensor in appropriate imaging position. Curious about the disparity in analyst results, this author concluded that in a multiple flash sequence beginning at 11:28, five debris surfaces must be fortuitously positioned. Alan D. Fischer, "Optics expert rebuts Waco standoff report," The Arizona Daily Star, Dec. 7, 2001.
27. Lena Klasen and Sten Madsen, op. cit, p. 47.
28. Maurice Cox, "Open Letter to Special Counsel Danforth," op. cit.
29. Lena Klasen and Sten Madsen, op. cit., pp. 81-82.
30. Carroll Lucas, "Declaration of Carroll L. Lucas," Isabel G. Andrade , et. al., v. Philip J. Chojnacki, et. al., United States District Court for the Western District of Texas, Waco Division, Case No. W-96-CA-139, 2000.
31. Jon Dougherty, "Expert supports Waco-video conclusions," WorldNetDaily, June 6, 2001.
32. David T. Hardy and Rex Kimball, op. cit., p. 115.
33. Ibid., p. 116.
34. Ibid., p. 123.
35. Ibid., p. 127.
36. Barbara Grant, "Rekindled Waco debate prompts IR test," Photonics Spectra, March, 2000, p. 24.
37. David T. Hardy and Rex Kimball, op. cit., p. 60.
38. Ibid, p. 62.
Barbara Grant is an electro-optical engineer specializing in the measurement of light. She has studied the Waco problem for the last several years.