Posted on 07/07/2011 8:55:49 PM PDT by TenthAmendmentChampion
According to database pioneer Michael Stonebraker, Facebook is operating a huge, complex MySQL implementation equivalent to “a fate worse than death,” and the only way out is “bite the bullet and rewrite everything.”
Not that it’s necessarily Facebook’s fault, though. Stonebraker says the social network’s predicament is all too common among web startups that start small and grow to epic proportions.
During an interview this week, Stonebraker explained to me that Facebook has split its MySQL database into 4,000 shards in order to handle the site’s massive data volume, and is running 9,000 instances of memcached in order to keep up with the number of transactions the database must serve. I’m checking with Facebook to verify the accuracy of those numbers, but Facebook’s history with MySQL is no mystery.
The oft-quoted statistic from 2008 is that the site had 1,800 servers dedicated to MySQL and 805 servers dedicated to memcached, although multiple MySQL shards and memcached instances can run on a single server. Facebook even maintains a MySQL at Facebook page dedicated to updating readers on the progress of its extensive work to make the database scale along with the site...
(Excerpt) Read more at gigaom.com ...
Junk it and run on DB2
Moving it to an MVS system instead? hmmm.... well the mainframes I know are still chugging along nicely with very little effort
Sql’s quite simple really — it’s when you start talking about grouping by statements and mix that with the different kinds of locks, hash indexing, join indexing, cylinder reads etc. that it gets complicated. And don’t get me started on RAID settings...
I vote for oracle as well. I”m not a coder, just a hardware person, but when I have to figure out errors on web based apps, I find oracle much easier to troubleshoot. I should say, send accurate information to the webmaster via their ora-XXXXXX error codes :)
Most likely. Back in my CICS system programming days a lot of the code seemed to be 20% doing the job and 80% making sure it failed gracefully. You don't bring down an on-line system unless something catastrophic has happened.
FB is probably 5% doing the job and 95% directing the code flow as changes to the security "model" are introduced.
Silly, when you consider that the languages you likely consider superior, run on VMs and OSs written in C++, or other languages you consider icky. It does all the heavy lifting and dangerous work that makes their slickness, safety, and even existence, possible.
I agree with that 100%
I do A LOT of contract work writing software. I tell people that unlike many programming languages where there are dozens of ways to do something that are all equally valid, there is usually only one RIGHT way to do a database, and all others are wrong
And a bad database works well when it is small, but as it grows you have serious problems NOT easy to fix.
I can see where Facebook grew from nothing and needed to work on freeware- but after their first billion or so they should have known to sink a huge chunk of that into redesign
>>They’d also be stuck with an oracle or sqlserver license fee out the wazoo.
Is that last a specialized finance term?
;-P
E.X.A.C.T.L.Y.
If you design the sh*t correctly the first time, rewrites aren’t required, even when the boss comes in at 4:59:59 on Friday and says “I gotta have it Monday!”
This may in fact be common, but it’s a symptom of lazy b@st@rds.
That and a serious lack of Oracle. (*checks ammo, ducks for cover*)
There’s no need to discuss brace styles. K&R is the correct style. Likewise VI is the best editor. (*jabs and ducks*) ;p
Very specialized. It’s covered in the advanced classes, usually not at the introductory level. :)
Hmmmm. Funny, I didn’t see this conflict in the movie.
- Someone is tasked with developing a small system now to demonstrate how the "imagined" system might look.
- The boss likes it and decrees that it should be rolled out for the entire group / division / etc. ASAP.
- Time is wasted for years fighting scalability and response time problems before the whole thing is scrapped (usually because the boss moved on and no one else thought it was worth maintaining).
I can’t count the number of times I’ve I seen that happen. I broke a VP of that nasty little habit one time by purposefully locking the DB down to one user and introducing a few sleep statements here and there in the code. When he came back from his smoke & mirrors magic show, he was all ears.
ISAM is such a smooth, fast little machine. I worked with it for years and loved it.
Sometimes you wonder why they throw 1,600 servers at something that could be accomplished by a few z-series mainframes. Imagine the space, electricity and manpower that’s being wasted.
I disagree. One of my company’s implemented an Oracle DB for a Fortune 100 company that handles all of their assets world wide and EVERY daily transaction from Purchase Orders to a Maintenance guy putting a drop of oil on a machine in Bum F Egypt; including inventory of parts and finished goods. Daily data dump is in the hundreds of gigs. Makes Facebook look like child’s play.
I've seen databases with tens of millions of records crushed not under the load itself, but under the poor design of the database. Redesign the database, and you can run it much faster on even slower hardware.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.