Skip to comments.
Facebook trapped in MySQL ‘fate worse than death’
Giga OM ^
| July 7, 2011
| Derrick Harris
Posted on 07/07/2011 8:55:49 PM PDT by TenthAmendmentChampion
According to database pioneer Michael Stonebraker, Facebook is operating a huge, complex MySQL implementation equivalent to a fate worse than death, and the only way out is bite the bullet and rewrite everything.
Not that its necessarily Facebooks fault, though. Stonebraker says the social networks predicament is all too common among web startups that start small and grow to epic proportions.
During an interview this week, Stonebraker explained to me that Facebook has split its MySQL database into 4,000 shards in order to handle the sites massive data volume, and is running 9,000 instances of memcached in order to keep up with the number of transactions the database must serve. Im checking with Facebook to verify the accuracy of those numbers, but Facebooks history with MySQL is no mystery.
The oft-quoted statistic from 2008 is that the site had 1,800 servers dedicated to MySQL and 805 servers dedicated to memcached, although multiple MySQL shards and memcached instances can run on a single server. Facebook even maintains a MySQL at Facebook page dedicated to updating readers on the progress of its extensive work to make the database scale along with the site...
(Excerpt) Read more at gigaom.com ...
TOPICS: Business/Economy; Computers/Internet
KEYWORDS: bsd; facebook; linux; mysql; opensource; socialnetwork; unix
Navigation: use the links below to view more comments.
first previous 1-20 ... 61-80, 81-100, 101-120, 121-127 last
To: antiRepublicrat
Sounds like someone finally cottoned onto what Honeywell did on their drive systems under CP-6.
121
posted on
07/08/2011 1:23:53 PM PDT
by
NVDave
To: NVDave
I don’t really like RAID 6 though. For a small installation of five drives, you might as well add one more and get RAID 10. for larger installations the big problems comes in drive rebuild times. With very large drives, the rebuild of one drive of a RAID 5 can take so long the risk of failure during the rebuild becomes more and more significant. RAID 6 just pushes the problem off a bit.
To: antiRepublicrat
ok — I work with datawarehouses so most of our stuff is RAID 1 or sometimes 5
123
posted on
07/08/2011 1:44:48 PM PDT
by
Cronos
( W Szczebrzeszynie chrzaszcz brzmi w trzcinie I Szczebrzeszyn z tego slynie.)
To: antiRepublicrat
A growing number of shops are dumping Wintel server farms (and even IBM RS6000 type farms) for Linux instances running on mainframes.
124
posted on
07/08/2011 2:31:02 PM PDT
by
djf
("Life is never fair...And perhaps it is a good thing for most of us that it is not." Oscar Wilde)
To: antiRepublicrat
You have users, threads, posts, mails and the connections between them.
LOL
125
posted on
07/08/2011 6:31:50 PM PDT
by
andyk
(Interstate != Intrastate)
To: mad_as_he$$
Feel free to disagree, but I think you need to improve your knowledge on Facebook's internal systems. There are several papers/preso off of
infoq.com and
highscability.com if you're so inclined. Both sites also have architecture information on other web scale companies.
I'm sure at one time hundreds of gigs per day was impressive. A year ago, Facebook had roughly
130TB of
logs per day. I'm sure that's gone up.
126
posted on
07/10/2011 10:22:21 PM PDT
by
tfecw
(It's for the children)
To: tfecw
Once controlled structure is achieved housekeeping is pure execution. I think the FB problem is that it was started by a dorm room operation not professional computer guys and then never upgraded to deal with large amounts of data. I have two guys that spend all day every day looking at architecture and DB design. They are worth their weight in gold since they are true system guru's. FB seem to have missed that.
Probably didn't make my point as clear as I could of. In data management foundation is everything and worth the initial pain. We start a project today with a very large customer still trying to run a relatively complex operation on Access of all things. Our biggest obstacle is the IT Dept that wrote the application years ago and has been nursing it all this time. Should be real interesting in a painful sort of way!
Navigation: use the links below to view more comments.
first previous 1-20 ... 61-80, 81-100, 101-120, 121-127 last
Disclaimer:
Opinions posted on Free Republic are those of the individual
posters and do not necessarily represent the opinion of Free Republic or its
management. All materials posted herein are protected by copyright law and the
exemption for fair use of copyrighted works.
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson