Windows: Index.dat, the hidden history file...

1stFreedom

[1stFreedom]

Folks, I bet you didn't know there is a "super secret" collection of files on your computer that contains all the websites you have ever visited.

Even if you think you have cleand out your surfing history, you are wrong. Windows stores the data in index.dat file and leaves the data there EVEN if you clean out the history, temporary internet folders, and cookies.

(If you use Outlook or Office products, that history is also stored -- even deleted messages!.)

While I'm a big advocate of "if you have nothing to hide don't worry" I'm also an advocate of keeping my pc free of my web visits, e-mails, etc.

I deleted my index.dat files and regained a whopping 28mb!!

(Keep in mind that most employers keep track, so some degree, of sites you have visited. Deleting index.dat files will only clean out your PC. Don't think you can visit forbidden sites and get away with it.)

You can't just find or delete these files from Windows easily -- they are hidden. On your favorite search engine do a search on Windows and index.dat. You'll find plenty of utilities that will clean these files out for you.

[goldylight]

it is in a few places - not only there. do a search for index.dat. you can open them up with a text editor - interesting to see what is in there. it will also vary due to your operating system.

[McGruff]

Be carefull before deleting these files.


What is an index.dat file?

There are three index.dat files residing in the following locations:

Cookies\index.dat

History\History.IE5\index.dat

Temporary Internet Files\Content.IE5\index.dat

index.dat files keep basically a copy of whatever there is in that folder and they continue keeping those records even after you have deleted the contents of those folders.


Why can't I find index.dat files on my hard disk?

Because index.dat files are hidden by default. However if you want to make the index.dat files visible, you can do that in Windows explorer folder options.


Can I delete index.dat files?

You can not delete index.dat files as long as Windows is running. That's why vendor's program needs to restart your PC in order to delete index.dat files.

Although as soon as Windows starts, new clean index.dat files will be created, ready to record your steps.


Does deleting index.dat files affect my PC in any way?

Deleting index.dat files is quite safe and besides freeing up some disk space has no other effects.


Is it possible to delete Cookies index.dat but keep other index.dat files?

No, because the contents of index.dat files are in close relationship with each other. Thus generally to delete the contents of any index.dat file, you have to delete all index.dat files at the same time.


How can I know that an index.dat file is clean?

The size of a clean Cookies index.dat file is 16 KB.

The size of a clean History index.dat file is also 16 KB.

When clean the size of Temporary Internet Files (Cache) index.dat file is 32 KB.

[widgysoft]

well he's right it does store tons of information on what sites you've visted.

an example from my index.dat

ð­http://www.freerepublic.com/focus/news/825585/posts posts[1] ð­HTTP/1.1 200 OK
Transfer-Encoding: chunked
Content-Type: text/html; charset=ISO-8859-1

~U:administrator

nothing unusual about that. I say leave it.

[WSGilcrest]

Lots of 3rd party stuff out there that say they will clean it. ----------------------------------------------------------

Index.dat--What is Index.dat File ?

---

Index.dat are files hidden on your computer that contain all of the Web sites that you have ever visited. Every URL, and every Web page is listed there. Not only that but all of the email that has been sent or received through Outlook or Outlook Express is also being logged. The file names and locations depend on what version of Internet Explorer you have. If you are running IE version 4.0 or above, the file name is "index.dat".  Microsoft has not supplied an adequate explanation as to what these files are for or why they have been hidden so well.

According to Microsoft, these files are used to cache visited Web sites to help speed up the loading of Web pages in Internet Explorer. Obviously this cannot be the case because when you clear the Temporary Internet Files the "index.dat" files remain behind and continue to grow. If you delete or clear the Temporary Internet Files, there is absolutely no need to index the URL cache because those files no longer exist.

On a Windows 9x computer these files are located in the following locations:

\WINDOWS\Cookies\index.dat
\WINDOWS\History\index.dat
\WINDOWS\Temporary Internet Files\index.dat
\WINDOWS\Cookies\index.dat
\WINDOWS\History\index.dat
\WINDOWS\Temporary Internet Files\index.dat

 In Windows 2000 and Windows XP there are several "index.dat" files in these locations:

\Documents and Settings\<Username>\Cookies\index.dat

\Documents and Settings\<Username>\Local Settings\History\History.IE5\index.dat

\Documents and Settings\<Username>\Local Settings\History\History.IE5\MSHist012001123120020101\index.dat
\Documents and Settings\<Username>\Local
 Settings\History\History.IE5\MSHist012002010720020114\index.dat

\Documents and Settings\<Username>\Local
 Internet Files\Content.IE5\index.dat

Index.dat files can be very hard to find. If you are in Windows, even with "Show hidden files and folders" enabled, index.dat files are not visible and cannot be found if you do a search for index.dat files. The reason that these files are so invisible is that they are not just hidden, they have been designated as "system" files. System files and folders are treated differently in DOS and Windows and are effectively cloaked from casual searches.

[demlosers]

So far I've found three index.dat files:

C:\WINDOWS\PCHEALTH\HELPCTR\OfflineCache 78kb

C:\Documents and Settings\user\Cookies 32kb

C:\Documents and Settings\user\UserData 32kb

Not too big

[demlosers]

I stop my search before it finished...

[general_re]

Oh, jeez. Come on. "Super secret"? Can we get a bit more tinfoilish?

"index.dat" is exactly what it sounds like - an index of IE's browser cache and associated cookies, built in order to speed up cache accesses. It's hardly "secret", and hardly worth the paranoia being engendered here....

[stuartcr]

In the days of DOS, 98 and before, you could just delete it from there, windows will make it new, when it needs it. Haven't tried it in 2000 or xp.

[socal_parrot]

From About.com-

Microsoft's Super Hidden Files

There are some files that defy detection on your computer

This article started out to be about another topic, but when I discovered the information about these super hidden Microsoft files, I had to write about it. I suppose that many of you already know about these files but this was the first I heard about it and I figured that if I didn't know about them, many of you didn't either.

Hidden on your computer are some files that contain all of the Web sites that you have ever visited. Every URL, and every Web page is listed there. Not only that but all of the email that has been sent or received through Outlook or Outlook Express is also being logged. The file names and locations depend on what version of Internet Explorer you have. If you are running IE version 4.0 or above, the file name is "index.dat". If you are running or have ever run IE prior to version 4.0 there are two files. One is named Mm256.dat and the other is Mm2048.dat. Microsoft has not supplied an adequate explanation as to what these files are for or why they have been hidden so well.

According to Microsoft, these files are used to cache visited Web sites to help speed up the loading of Web pages in Internet Explorer. Obviously this cannot be the case because when you clear the Temporary Internet Files the "index.dat" files remain behind and continue to grow. If you delete or clear the Temporary Internet Files, there is absolutely no need to index the URL cache because those files no longer exist.

On a Windows 9x computer these files are located in the following locations:

\WINDOWS\Cookies\index.dat
\WINDOWS\History\index.dat
\WINDOWS\Temporary Internet Files\index.dat
\WINDOWS\Cookies\index.dat
\WINDOWS\History\index.dat
\WINDOWS\Temporary Internet Files\index.dat

As I stated earlier, these files can be very hard to find. If you are in Windows, even with "Show hidden files and folders" enabled, these files are not visible and cannot be found if you do a search for these files. The reason that these files are so invisible is that they are not just hidden, they have been designated as "system" files. System files and folders are treated differently in DOS and Windows and are effectively cloaked from casual searches.

I am currently running Windows XP Pro and since there is no underlying DOS core, these rules do not apply and therefore with "Show hidden files and folders" enabled, I can do a search on these files and find them with no trouble. In Windows XP there are several "index.dat" files in these locations:

\Documents and Settings\Default User\Cookies\index.dat
\Documents and Settings\Default User\Local Settings\History\History.IE5\index.dat
\Documents and Settings\Default User\Local Settings\Temporary Internet Files\Content.IE5\index.dat
\Documents and Settings\Default User.WINDOWS\Cookies\index.dat
\Documents and Settings\Default User.WINDOWS\Local Settings\History\History.IE5\index.dat
\Documents and Settings\Default User.WINDOWS\Local Settings\Temporary Internet Files\Content.IE5\index.dat
\Documents and Settings\Bill\Application Data\Microsoft\Office\Recent\index.dat
\Documents and Settings\Bill\Cookies\index.dat
\Documents and Settings\Bill\Local Settings\History\History.IE5\index.dat
\Documents and Settings\Bill\Local Settings\History\History.IE5\MSHist012001123120020101\index.dat
\Documents and Settings\Bill\Local Settings\History\History.IE5\MSHist012002010720020114\index.dat
\Documents and Settings\Bill\Local Settings\History\History.IE5\MSHist012002011420020121\index.dat
\Documents and Settings\Bill\Local Settings\History\History.IE5\MSHist012002012120020128\index.dat
\Documents and Settings\Bill\Local Settings\History\History.IE5\MSHist012002012820020129\index.dat
\Documents and Settings\Bill\Local Settings\History\History.IE5\MSHist012002012920020130\index.dat
\Documents and Settings\Bill\Local Settings\Temporary Internet Files\Content.IE5\index.dat
\Documents and Settings\Bill\UserData\index.dat

There is a very detailed explanation of these files and how Microsoft has managed to hide them so well by a person who calls himself "The Riddler". He goes into great detail and provides a lot of information on these super hidden files.

If having these files makes you feel paranoid (and who wouldn't) there is a good little program called "Spider" that can find these files and delete them for you. There is also another good program called PurgeIE that can find and delete these files. Spider is freeware, PurgeIE is shareware and can be registered for $14.95.

[Captain Beyond]

bump for later read

[Hot Tabasco]

Turned on my computer last night and all my personal "Favorites" were gone. Ive got Window's 2000. Any ideas?

[McGruff]

Index.dat Files

Ah, the index.dat files. What are they? What purpose do they serve?

Simple as this: the index.dat files are used by Internet Explorer mainly in the Temporary Internet Files folder and the History folder.

In the Temporary Internet Files folder, Internet Explorer uses the index.dat file to keep track of each web site you visit and each file associated with each web site. That way, when it needs to retrived cached pages, it can do so by reading the information in the index.dat file for a particular site.

A single entry in the index.dat file looks like this when viewed in a plain text editor:

~U:username
URL . pw—.RÆÁ. c. ` h . ..¤ A. ´  h,ú.. h,ú. http://www.tweakie.com/default.htm default[1].htm HTTP/1.1 200 OK
Content-type: text/html
Page-Completion-Status: Normal
Page-Completion-Status: Normal


As you can see, even with some of the gibberish, it is quite easy to extract the site and page that was viewed. In most cases, this can be done for every page you have ever visited. In the best case, for you, this can be done for only the pages you have visited over the last 6 months or so.

How can this be? Why would Internet Explorer need this data after 6 months or more? Is it still there if I delete my Temporary Internet Files via Internet Explorer?

The simple answers are: 1) Who knows?, 2) Due to page caching and 3) Yes!

The complete answers are:
1) We still don't know exactly
2) Since Internet Explorer caches web pages so they can be displayed more quickly without the need to download them again and because page designers can set their own expiration dates to tell Internet Explorer how often it should re-download the entire page, this information is retained.
3) Yes, the information in the index.dat file remains even if the files for ANY given site are deleted. Why? We don't know. Ask Microsoft, they won't tell you and have never explained it when the media has spoken to us and then to them. The fact of the matter is that if you think you have deleted all traces of web sites you have visited by deleting the Temporary Internet Files, think again.

Due to our knowledge of these files, we have been contracted, on occasion, to retrieve information from them by companies seeking the knowledge of what their employees have been viewing on company systems even after the Temporary Internet Files are deleted. In one case, a worker was fired from a school for viewing inappropriate material and, in another case, a worker was brought up on insider trading charges filed with the SEC due to online trading he was doing at his company.

Keep in mind, that only someone with direct access to your computer can gain access to these files.

No web sites you need to hide you say? Well, there is also a performance consideration. Each time Internet Explorer loads a page, it scans the index.dat file. This file, since it is never emptied, keeps growing larger and larger until it is many megabytes in size. Logic says that the larger the file, the longer it takes to read it. Microsoft seems to have alleviated most crashes in Internet Explorer that were occurring as the index.dat file grew too large, but Internet Explorer can still drag a bit when the file gets to big.

In the History folder, the index.dat file is used in very much the same way, except that it is used to track links you have clicked on so Internet Explorer can display them as visited. Also, there is one main index.dat file as well as one in each of the daily folders for each day's History.

The same problems are inherent to these index.dat files as the one in the Temporary Internet Files

[jonascord]

BUMP FOR LATER

[general_re]

Oh, well, if you can't trust some anonymous guy on the internet calling himself "The Riddler", who can you trust these days?

[spodefly]

While you are at it, get rid of those goofy 'autoexec.bat' and 'config.sys' files too ... It will speed the start-up dramatically.








{of course, your machine will start-up only to a flashing cursor, but it will be fast!}

[tophat9000]

bump

[Psycho_Bunny]

lololololololol

How "Super Hidden" can they be if you can find them?

If you really wanted to hide files, you wouldn't just flick the SH attrib switches.

[is_is]

bump

[kever]

OH GOD! I hope my boss doesn't find out I've been browsing FREEREPUBLIC.COM har har

[McGruff]

Here's a link to some freeware Internet Cleanup programs. One called Spider specifically says it cleans index.dat files.

As always proceed with caution.