Replies

Some when we “permanently” delete....They aren’t really gone, gone?

With most filesystems, no. Deleting a file doesn't actually do anything at all to the actual data associated with a file. A filesystem is like a book with an index. The index points to the individual pages in the book. When you delete a file it is that index entry that is deleted. When the pointer to that page is deleted, the page is marked as 'free', so you can now write new data to it. Until new data is actually written to that page, the data is still there, and proper forensics can recover it.

I do not know exactly how it works these days with NTFS or whatever it is that windows uses for a filesystem, but back in the old days before we actually had tools that would allow you to recover files, I had a program that would actually let you directly edit individual sectors on a disk. I used that to learn a bit about how the FAT tables worked. When you would delete a file, not only was the filespace not deleted, but the 'index' pointer wasn't actually deleted either. What happened instead was that the first character of the filename was changed from whatever it was to the hexadecimal value of "E5". So, to 'undelete' a file, if you could edit the disk, you could change that first character back to a standard ASCII value like '61' (which is an 'a'), and the file would be magically undeleted. You could also do some neat things like change that first character to an unprintable character, so that if you didn't know what it was, it would make it hard to access that file.

Modern filesystems are a bit more complex than FAT was back in the day, but from what I understand, all FAT derivatives, like FAT16, FAT32, and EXFAT which are typically used on memory cards and thumb drives are essentially the same.

Another thing about the way files are allocated is that they are actually laid out in chunks of a standard size. The size of those chunks is fixed when you format the drive, so it makes the whole 'book with pages' actually a closer analogy than you might think. Let's say that you allocation unit is 1024 bytes. That means that if you create a file that has nothing in it but the text "hillary is a liar" (18 bytes), it is still going to take up 1024 bytes of your disk space. So, if you have, say 100,000 tiny files of a few bytes each, they could easily take up more space than a single one megabyte file.

So, what about all that extra unused space at the end of the file? Well it is essentially wasted. There are security ramifications of this. Let's say you had a file that used up all of the allocated space.

AAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAA

Now, we change the file, but when we updated the file the data wasn't as big as the previous file. You might end up with something like this that is actually on your disk...

BBBBBBBBBBBBBBBBBBBB
BBBBBBBBBBBBBBBBBBBB
BBBBBBBBBBBBBBBBBBBB
BBBBBBBBBBBBBBAAAAAA
AAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAA

When the new file is written, it doesn't overwrite everything in the block, so some of the data that you previously had in it that you thought you had deleted is no longer there actually still remains. THere are forensic tools that can pull that data from the end of the blocks out as well. Yeah, it's going to be fragmented and incomplete data, but sometimes that's all you need.

So, to get back to the topic of the thread, a program that securely deletes a file should also zero out that slack space at the end of the allocation block as well as the file itself. Similarly, a program designed to wipe "free" space on the drive should also get those unused fragments at the end of the blocks as well.

All of this is probably more than you needed, or wanted to know, but hopefully someone will find if of interest. :-)