Hmm. This guy should give me a call. We were putting 100+ million records on Linux Slackware systems using raw devices in 1995. Have since invented something that will easily hold billions of records that can span multiple machines across any network.
I use ext4 on all of my Linux distros, if possible. It seems to me that SSDs are the next phase of the evolution of storage. Having recently purchased my first 1 TB SATA disk for my new gaming rig, I decided to run Windows 7’s system experience test on my system. I scored 7.6, 7.6, 7.7, and 7.7 respectively on each of the tests (CPU, memory, DirectX, and video), but my hard disk dumped my score down to a 5.9 (MS uses the lowest score as the final). I was shocked, to say the least, but my previous system had a 150 GB SATA disk at 10K RPM rotational speed and netted me a 6.5 on the same test.
Size truly does matter, but interface bandwidth and operating system disk operations appear to be the primary concerns.
Why wouldn’t you use a database?
http://en.wikipedia.org/wiki/ZFS
I thought that Apple almost put this into 10.5 (maybe), but had some license issues...Not really sure, but it is good to know that folks are in front of this.
Regardless of whether you’re on an array, Windows with NTFS starts dying with only 20,000 or so files in a single folder. You’re sure to get a lock-up with half a million files.
Hmm. This guy should give me a call. We were putting 100+ million records on Linux Slackware systems using raw devices in 1995. Have since invented something that will easily hold billions of records that can span multiple machines across any network.
Hmmm... how many files do I have on my computer...
$ sudo find / -print | wc -l
605730
That's a lot of files IMO for a simple desktop computer, but nowhere near what they are talking about.
I've dealt with directories at work with 100k+ files in them (as a result of really stupid programmers in this case), and it's not pretty when you need to do cleanup there. Thank God for xargs!
File proliferation is actually a serious issue. Desktop users, especially those that have never figured out how to properly organize files can quickly get into a state where they can't find files they are looking for, even if they know it's somewhere on the computer. Your average user doesn't know anything about how to efficiently organize directories and subdirectories. Heck, I sometimes have problems with it myself, and find myself going back through and reorganizing things periodically just to keep things straight.