Apple, FBI, and the Burden of Forensic Methodology

Recently, FBI got a court order that compels Apple to create a forensics tool; this tool would let FBI brute force the PIN on a suspect’s device. But lets look at the difference between this and simply bringing a phone to Apple; maybe you’ll start to see the difference of why this is so significant, not to mention underhanded.

First, let me preface this with the fact that I am speaking from my own personal experience both in the courtroom and working in law enforcement forensics circles since around 2008. You can find my CV here. I’ve testified as an expert in three cases in California, and many others have pleaded out or had other outcomes not requiring my testimony. I’ve spent considerable time training law enforcement agencies around the world specifically in iOS forensics, met LEOs in the middle of the night to work on cases right off of airplanes, gone through the forensics validation process and clearance processes, and dealt with red tape and all the other terrible aspects of forensics that you don’t see on CSI. It was a lot of fun but was also an incredibly sobering experience, as I have not been trained to deal with the evidence (images, voicemail, messages, etc) that I’ve been exposed to like LEOs have; my faith has kept me grounded. I’ve developed an amazing amount of respect for what they do.

For years, the government could come to Apple with a warrant and a phone, and have the manufacturer provide a disk image of the device. This largely worked because Apple didn’t have to hack into their phones to do this. Up until iOS 8, the encryption Apple chose to use in their design was easily reversible when you had code execution on the phone (which Apple does). So all through iOS 7, Apple only needed to insert the key into the safe and provide FBI with a copy of the data.

This service worked like a “black box”, and while Apple may have needed to explain their methods in court at some point, they were more likely considered a neutral third party lab as most forensics companies would be if you sent them a DNA sample. The level of validation and accountability here is relatively low, and methods can often be opaque; that is, Apple could simply claim that the tech involved was a trade secret, and gotten off without much more than an explanation. An engineer at Apple could hack up a quick and dirty tool to dump disk, and nobody would need to ever see it because they were providing a lab service and were considered more or less trade secrets.

Now lets contrast that history with what FBI and the courts are ordering Apple to do here. FBI could have come to Apple with a court order stating they must brute force the PIN on the phone and deliver the contents. It would have been difficult to get a judge to sign off on that, since this quite boldly exceeds the notion of “reasonable assistance” to hack into your own devices. No, to slide this by, FBI was more clever. They requested that Apple developed a forensics tool but did not do the actual brute force themselves.

This was apparently enough for the courts to look past the idea of “reasonable assistance”, however there are some unseen caveats that are especially dangerous here. What many haven’t considered is the significant difference – in the legal world – between providing lab services and developing what the courts will consider an instrument.

An instrument is the term used in the courts to describe anything from a breathalyzer device to a forensics tool, and in order to get judicial notice of a new instrument, it must be established that it is validated, peer reviewed, and accepted in the scientific community. It is also held to strict requirements of reproducibility and predictability, requiring third parties (such as defense experts) to have access to it. I’ve often heard Cellebrite referred to, for example, as “the Cellebrite instrument” in courts. Instruments are treated very differently from a simple lab service, like dumping a phone. I’ve done both of these for law enforcement in the past: provided services, and developed a forensics tool. Providing a simple dump of a disk image only involves my giving testimony of my technique. My forensics tools, however, required a much thorough process that took significant resources, and they would for Apple too.

The tool must be designed and developed under much more stringent practices that involve reproducible, predictable results, extensive error checking, documentation, adequate logging of errors, and so on. The tool must be forensically sound and not change anything on the target, or document every change that it makes / is made in the process. Full documentation must be written that explains the methods and techniques used to disable Apple’s own security features. The tool cannot simply be some throw-together to break a PIN; it must be designed in a manner in which its function can be explained, and its methodology could be reproduced by independent third parties. Since FBI is supposedly the ones to provide the PIN codes to try, Apple must also design and develop an interface / harness to communicate PINs into the tool, which means added engineering for input validation, protocol design, more logging, error handling, and so on. FBI has asked to do this wirelessly (possibly remotely), which also means transit encryption, validation, certificate revocation, and so on.

Once the tool itself is designed, it must be tested internally on a number of devices with exactly matching versions of hardware and operating system, and peer reviewed internally to establish a pool of peer-review experts that can vouch for the technology. In my case, it was a bunch of scientists from various government agencies doing the peer-review for me. The test devices will be imaged before and after, and their disk images compared to ensure that no bits were changed; changes that do occur from the operating system unlocking, logging, etc., will need to be documented so they can be explained to the courts. Bugs must be addressed. The user interface must be simplified and robust in its error handling so that it can be used by third parties.

Once the tool is ready, it must be tested and validated by a third party. In this case, it would be NIST/NIJ (which is where my own tools were validated). NIST has a mobile forensics testing and validation process by which Apple would need to provide a copy of the tool (which would have to work on all of their test devices) for NIST to verify. NIST checks to ensure that all of the data on the test devices is recovered. Any time the software is updated, it should go back through the validation process. Once NIST tests and validates the device, it would be clear for the FBI to use on the device. Here is an example of what my tools validation from NIJ looks like: https://www.ncjrs.gov/pdffiles1/nij/232383.pdf

During trial, the court will want to see what kind of scientific peer review the tool has had; if it is not validated by NIST or some other third party, or has no acceptance in the scientific community, the tool and any evidence gathered by it could be rejected.

Apple must be prepared to defend their tool and methodology in court; no really, the defense / judge / even juries in CA will ask stupid questions such as, “why didn’t you do it this way”, or “is this jail breaking”, or “couldn’t you just jailbreak the phone?” (i was actually asked that by a juror in CA’s broken legal system that lets the jury ask questions). Apple has to invest resources in engineers who are intimately familiar with not only their code, but also why they chose the methodology they did as their best practices. If certain challenges don’t end well, future versions of the instrument may end up needing to incorporate changes at the request of FBI.

If evidence from a device ever leads to a case in a court room, the defense attorney will (and should) request a copy of the tool to have independent third party verification performed, at which point the software will need to be made to work on another set of test devices. Apple will need to work with defense experts to instruct them on how to use the tool to provide predictable and consistent results.

In the likely event that FBI compels the use of the tool for other devices, Apple will need to maintain engineering and legal staff to keep up to date on their knowledge of the tool, maintain the tool, and provide testimony as needed.

In other words, developing an instrument is far more involved than simply dumping a phone for FBI, which FBI could have ordered:

This far exceeds the realm of “reasonable assistance”, especially considering that Apple is not a professional forensics company and has no experience in designing forensic methodology, tools, or forensic validation. FBI could attempt to circumvent proper validation by issuing a deviation (as they had at one point with my own tools), however this runs the risk of causing the house of cards to collapse if challenged by a defense attorney.

So in light of the demand of sound forensic science, the Department of Justice’s outrageous arguments seem quite inaccurate.

Quite the contrary, unless Department of Justice is asking Apple to completely ignore sound forensic science, and simply pump out a reckless (and possibly harmful) hacking tool, it would seem that false statements are being made to the court. Or perhaps they’re attempting to skirt the reality of this by using the verbiage, “after its purpose”, which requires disseminating it outside of Apple, as well as opening it up to work on other devices, and thereby relinquishing custody of it.

In the same vein, you’ll also notice that in demanding a tool, FBI has sneakily ensured that a more “open” copy of the software will have to be released (that will work on other devices) in order for it to be tested, validated, and re-tested by a defense team. This guarantees that the hacking tool FBI is forcing Apple to write will be out in the public, where it will be in the hands of multiple agencies and private attorneys.

Not only is Apple being ordered to compromise their own devices; they’re being ordered to give that golden key to the government, in a very roundabout sneaky way. What FBI has requested will inevitably force Apple’s methods out into the open, where they can be ingested by government agencies looking to do the same thing. They will also be exposed to private forensics companies, who are notorious for reverse engineering and stealing other people’s intellectual property. Should Apple comply in providing a tool, it will inevitably end up abused and in the wrong hands.

But will this case ever need to see a court room? Absolutely, they’ve already admitted they’re following leads and looking at (or at lest for) other people. If a relative or anyone else involved is prosecuted, these tools will come up in court. Outside of this one case, you’re no doubt aware of the precedent this sets, and the likelihood this tool won’t be used once, but many times, each having to establish courtroom acceptance in different jurisdictions, different defense challenges, giving the software to more parties for analysis and reproducible results, and so on.

You’re asking the wrong question. Consider this, even if a suspect never went to court, we’re talking about practicing sound forensic science. Everything I’ve outlined in this article is consistent with best practices in the field. For anyone to be okay with a simple ugly hack job instead of a forensics tool would set an ugly precedent of skirting sound science and methodology in handling of evidence. This would undoubtedly do damage to the reputation of the forensic process, and lower the bar on all such standards. In other words, the reputation of forensic science is more important than whether or not this case will ever see a courtroom.

“So how many devices have performed a forensic examination on? How many forensic tools have you developed and had to defend in court? Please, give us your qualifications to provide an educated opinion on the article.”

And this has what to do with the necessary availability of records and all materials needed in both civil and criminal trials, for discovery or for use by, oh, say, forensics? The 4th amendment is the basis of our freedom. If we can not have access to materials relevant to a case through discovery, we can not defend ourselves.

Sorry, you’re talking to a Contituionalist. It doesn’t matter to me what your credentials are, you are just another jagov that wants to obfuscate and deflect. Not interested.

You and your little band of Myopics would do well to lift your sights to something much larger than your shares in Apple. I give you all the benefit of greed as motivation rather than the enormous stupidity that would indicate a willingness to undermine our Constitution to serve your ridiculous worship of “right to privacy with no reserve.”

Do you understand freedom? You are all advocating Anarchy.

I also think the AWA is not the way to do this- forcing a third party to do that much to help.
Subpoenaing Apple’s code and keys so the FBI could make it’s own ‘instrument’ would be the ‘normal’ way to handle this I believe- but that has obvious problems!
I hope everyone works to address all the problems.

My own opinion is to treat refusing to unlock encryption when a warrant is presented the same as destruction of evidence, and to require ‘encryptors’ to open their encryption at their own expense in cases of deceased owners.

Encryption isn’t going away, I hope the Fourth Amendment isn’t either.

Sorry, you’re talking to a Contituionalist.

I doubt that. IF you were, you would recognize that the Constitution does not permit the government to force an individual or a company to create a new product that does not currently exist. If the tool already existed, then they could force them to turn it over. But to demand that they create something brand new, and in the process severely damage the value of one of their flagship products, is not something you can find in the Constitution.

By the way, I don't own any shares of Apple - I just believe in the Constitution. And I have enough experience in the field to know that what the author posted is accurate.

You are all advocating Anarchy.

No, you are advocating tyranny. The Founders knew when they wrote the Bill of Rights that the protections they put in there meant that sometimes, the bad guys would go free. To them, that was the price they were willing to pay to prevent the government from being able to oppress them.

“‘unbreakable’ encryption that can be opened” (I notice you placed unbreakable in ironic quotes, indicating even you know it would now be very breakable....lol)

But while you babble on about openable unbreakable encryption to be sure you are safe in bubble world, consider something;
The same FBI demanding this, has scrubbed all of their anti-terror training materials of anything remotely suggesting islam is a terror breeding ground. They supplied weapons to the Cartels to undermine second amendment rights. They have not charged Hillary, nor the IRS director who used office to harass TEA associated citizens.
They refused to look at the female terrorists facebook page “out of respect for her privacy”.
They did not act on the Russian warnings about the Boston bombers. They designated mentioning the constitution, being a vet, having a Gadsden flag or a Ron Paul sticker as an indicator of terrorism.
They went to the middle east to charge the Blackwater guys despite having ZERO legality to do so. A million and a half moslems have been brought in since 9/11 and they want another 300k.

And you claim we are unsafe because of an Iphone. Bright.

“No, you are advocating tyranny. The Founders knew when they wrote the Bill of Rights that the protections they put in there meant that sometimes, the bad guys would go free. To them, that was the price they were willing to pay to prevent the government from being able to oppress them.”

BWAHAHAHAHAHAHAHAHA.

WTF? Cut and paste from liberalsmakingshitup.com?

Sorry Bud, you are completely at a loss about how the rules of discovery are meant to work and so hung up on criminal intent and/or paranoia about SOMETHING you can’t see the forest for the trees. REAL freedom in this country hinges on the ability to do commerce, regulated through the LAW. The LAW run thngs and if it is not protected through the Constitution you and all the little pissants worried about protecting their little nasty secrets are lost anyway. This is high stakes, Junior.

This little ploy of Apple would be great for your short term stock portfolio, but would do more to kill small business than you could possibly imagine. If you knew how woefully ignorant of this aspect of commerce you truly were, you would die of shame. Go ahead, help make the world’s biggest Corporations your masters. You’ll love it.

“If we can not have access to materials relevant to a case through discovery, we can not defend ourselves.”

They do have access. They have an iphone they can go to NSA headquarters and go right to work. They can search it all they want. But a search warrant doesn’t mean you get success. If they have a search warrant to search the safe in my house, I don’t have to unlock my door or lock up the dogs. I don’t have to disable the alarm. The cops cant call the Safe manufacturer and demand they come down and figure out how to get them in.
The cops can get out saws and blowtorches and drills. They might call lightfingers Vinnie the safecracker.
The warrant allows all that.

And you say we can’t be safe? Sure we can. You don’t import moslems by the hundreds of thousands from war zones and refuse to even watch them.

You accuse us of wanting anarchy. That is what the government has given us. And to fix it, you want a healthy dose of fascism now. No dice sport.

Apple says it can meet the writs’ demands very cheaply LOL!

This Author does NOT say Apple is lying or stupid. . . but you do. You claim that Apple has said it can meet the writ's (there is no such thing, there is a Court Order made by Magistrate Judge Sheri Pym, which was actually written by the lawyers for the FBI, but it is not the "writ's" demands) very cheaply. Apple has actually said it would take up to ten engineers, a documentarian, a team leader, up to a month's time JUST to get to the point where they can start TESTING any new version of what the Order has demanded. Jonathan Zdziarski, because of his expertise, laid out WHY it would take that long and why it would be expensive and why it would require a documentarian. He also showed what expenses would ensue AFTER Apple had created such an "instrument". . . and the validation it would require before anything found on it could be used in courts of record to gain a conviction. EXACTLY what I have outlined based on MY experience. He outlined the implied on-going costs associated with that. YOU conflate that with Zdziarski claiming Apple is lying? He validates Apple's claim that what the court is demanding is unduly onerous.

You've made this claim before and I've challenged you on it. . . which you've ignored. Where, exactly, did Apple say they could perform all this "very cheaply."

Put up, or shut up.

I'm still waiting in that other thread for you to respond to the same question.

By “like”, do you lean more toward the “get a warrant” angle or the “got a warrant” angle? Remember, the Founding Fathers were quite aware of cryptology, and did NOT include anything about empowering the government to compel others to break codes; near as I can tell, they would have responded to the current situation with “you got your warrant, you got the data, tough $#!^ if you can’t decrypt it.”

Well ya got me there, sport. I’m still talking to you.

Now you project your demonstrated ignorance on to me. Pathetic. I read the article you denigrated and pathetically lied about.

I find the ignorance coming from you BURNS. It is, frankly, mind numbingly, deeply painful to read, over and over again. Your absolute refusal to read this very informative article is just one more example of your desire to remain willfully ignorant. . . and then proudly LIE about what you claimed was in the article. You don't even have the honor, honesty or decency to admit you did not read what you claimed you did. LIAR. Yet you insist on inflicting your ignorance and resultant ignorant lies on all of us. Please stop. Get an education.

“This little ploy of Apple would be great for your short term stock portfolio, but would do more to kill small business than you could possibly imagine.”

Au contraire.

Implementing virtually unbreakable encryption is MUCH cheaper than implementing the “back door” you want. If nothing else, Apple is quite justified in _not_ consenting to FBI demands precisely because they’ll spend enormous sums cracking computer security for every judge rubber-stamping a warrant and/or trying to implement a “back door” that won’t be compromised in short order (and cleaning up the mess each & every time it is, which it will be).

“You claim the Founders’ Fourth Amendment is some kind of tyranny and don’t expect to be ignored as a fool”

You are an idiot in the field of the 4th amendment. They 4th amendment lets them seize the phone. They did. It lets the Government open and study it and use the information in it. The 4th does not require ANYONE outside of government to do anything on their behalf.

I know you aren’t too up to speed. But its 1799. Someone writes a letter about a plot to shoot Jefferson. The letter is intercepted with a search warrant. But it is opened and found to be written in a secret Masonic Dan Brown code, backwards, in a mirror.
The warrant allows the government to seize the letter and to set their codebreakers on it. The warrant -does not- mean someone who invented this cool secret code must come down and decipher it for the government.

The 4th amendment is beautifully written. You are the fool.

It’s about 150 grand- according to Apple: UP TO a ten man team with 6 to 8 programmers LESS THAN a month.

Who made YOU Apple's JOB accountant? I saw that response and told you it was a "Facturd" you pulled out of your nether sphincter. You don't know how much those software engineers are paid per month. You don't now what the testing will cost. You know NOTHING. . . so you impute that $150,000 is 'very cheaply" because you created the figures. Right. NO it is not right.

You can't have "unbreakable" encryption and have it breakable too. They are mutually exclusive.

Subpoenaing Apple’s code and keys so the FBI could make it’s own ‘instrument’ would be the ‘normal’ way to handle this I believe- but that has obvious problems! I hope everyone works to address all the problems.

I would be opposed to that approach as well. The security would be just as much compromised if not more. The FBI is NOT entitled to Apple's private property, especially without just compensation. That code is worth, at a modest estimate, $150 billion, and probably much more. Are you willing for the government to pay that for it?

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.