Roadrunner: 130,536 cores break the Petaflop barrier ( the world’s first hybrid supercomputer)

Armonk (NY) – IBM is prepared to deliver the Roadrunner supercomputer to the Department of Energy’s National Nuclear Security Administration (NNSA). The system was development over the past 18 months and is not only the first hybrid supercomputer using Cell processors, but also the first commercial system to exceed a performance of 1 PFlops.

Roadrunner is the NNSA’s third IBM-built supercomputer, adding to two IBM-built supercomputers at the Lawrence Livermore National Laboratory: BlueGene/L, completed in 2005, has 131,072 p5 processors and delivers a sustained performance of 280.6 TFlops. The 12,208 processor ASC Purple is estimated to provide a sustained performance of 75.8 teraflops.

The new Roadrunner, which is currently disassembled and will be shipped to the NNSA within 30 days, according to IBM, was originally described to deliver about 2.8x (3.5x peak: 1.6 TFlops ) the combined processing power of BlueGene/L and ASC Purple. Back in 2006, IBM said that the system would bet more than 16,000 Opteron processor cores and more than 16,000 Cell BE CPUs to deliver on the Petaflop goal.

The final system ended up with less processors – 13,896 Opteron cores (6948 dual-core processors) as well as 12,960 Cell BE chips with a total of 116,640 PPE and SPE cores for a combined 130,536 cores. The original design called for about 176,000 cores.

According to IBM, Roadrunner has the floating point performance of about 100,000 notebooks. To describe the horsepower of the system, IBM said that Roadrunner can do in one day what would take the entire population of the world 46 years on handheld calculators, assuming a pace of one second per calculation.

Other interesting facts of the supercomputer, which IBM said cost about $100 million, include a power consumption of about 3.9 Mwatts, as well as a structure that includes 80 TB of memory, 576 miles of fiber optic cable as well as 3456 tri-blades which can deliver 400 GFlops each.

Also noteworthy is that Roadrunner was supposed to become the flagship product of AMD’s Torrenza platform, a project which was pitched as an open AMD x86 platform that takes advantage of Direct Connect Architecture and the Hypertransport interface. However, we haven’t heard much about Torrenza anymore and we don’t expect AMD to make a big deal of it.

Fact Sheet & Background: Roadrunner Smashes the Petaflop Barrier

***********************EXCERPS*********************

Armonk, NY - 09 Jun 2008: In 2006, the Department of Energy’s National Nuclear Security Administration selected Los Alamos National Laboratory as the development site for Roadrunner and IBM as the computer’s designer and builder. Roadrunner, named after the New Mexico state bird, cost about $100 million, and was a three-phase project to deliver the world’s first “hybrid” supercomputer – one powerful enough to operate at one petaflop (one thousand trillion calculations per second). That’s twice as fast as the current No.1 rated IBM Blue Gene system at Lawrence Livermore National Lab – itself nearly three times faster than the leading contenders on the current TOP 500 list of worldwide supercomputers.

Roadrunner will primarily be used to ensure the safety and reliability of the nation’s nuclear weapons stockpile. It will also be used for research into astronomy, energy, human genome science and climate change.

Roadrunner is the world’s first hybrid supercomputer. In a first-of-a-kind design, the Cell Broadband Engine® -- originally designed for video game platforms such as the Sony Playstation 3® -- will work in conjunction with x86 processors from AMD®.

Made from Commercial Parts. In total, Roadrunner connects 6,948 dual-core AMD Opteron® chips (on IBM Model LS21 blade servers) as well as 12,960 Cell engines (on IBM Model QS22 blade servers). The Roadrunner system has 80 terabytes of memory, and is housed in 288 refrigerator-sized, IBM BladeCenter® racks occupying 6,000 square feet. Its 10,000 connections – both Infiniband and Gigabit Ethernet -- require 57 miles of fiber optic cable. Roadrunner weighs 500,000 lbs. Companies that contributed components and technology include; Emcore, Flextronics, Mellanox and Voltaire.

Custom Configuration. Two IBM QS22 blade servers and one IBM LS21 blade server are combined into a specialized “tri-blade” configuration for Roadrunner. The machine is composed of a total of 3,456 tri-blades built in IBM’s Rochester, Minn. plant. Standard processing (e.g., file system I/O) is handled by the Opteron processors. Mathematically and CPU-intensive elements are directed to the Cell processors. Each tri-blade unit can run at 400 billion operations per second (400 Gigaflops).

The machine was built, tested and benchmarked in IBM’s Poughkeepsie, N.Y. plant, home of the ASCI series of supercomputers the company built for the US government in the late 1990s. IBM’s site in Rochester, Minn. constructed the specialized tri-blade servers. Software development was led by IBM engineers in Austin, Texas and by researchers in IBM’s Yorktown Heights, N.Y. research lab. Roadrunner will be loaded onto 21 tractor trailer trucks later this summer when it is delivered to Los Alamos National Lab in New Mexico.

Roadrunner operates on open-source Linux software from Red Hat.

Energy Miser. Compared to most traditional supercomputer designs, Roadrunner’s hybrid format sips power (3.9 megawatts) and delivers world-leading efficiency – 376 million calculations per watt. IBM expects Roadrunner to place among the top energy-efficient systems later in June when the official “Green 500” list of supercomputers is issued.

IBM is developing new software to make Cell-powered hybrid computing broadly accessible. Roadrunner’s massive software effort targets commercial applications for hybrid supercomputing. With corporate and academic partners, IBM is developing an open-source ecosystem that will bring hybrid supercomputing to financial services, energy exploration and medical imaging industries among others.

Applications for Cell-based hybrid supercomputing include: calculating cause and effect in capital markets in real-time, supercomputers in financial services can instantly predict the ripple effect of a stock market change throughout the markets. In medicine, complex 3-D renderings of tissues and bone structures will happen in real-time, as patients are being examined.

How fast is a petaflop?
Roadrunner operates at speeds exceeding one petaflop -- one thousand trillion calculations per second -- or one million billion calculations per second; or one quadrillion calculations per second.

Lots of laptops. That’s roughly equivalent to the combined computing power of 100,000 of today’s fastest laptop computers. You would need a stack of laptops 1.5 miles high to equal Roadrunner’s performance.

It would take the entire population of the earth, -- about six billion – each of us working a handheld calculator at the rate of one second per calculation, more than 46 years to do what Roadrunner can do in one day.

In the past 10 years, supercomputer power has increased about 1,000 times. Today, just three of Roadrunner’s 3,456 Tri-blade units have the same power as the 1998 fastest computer. A complex physics calculation that will take Roadrunner one week to complete, would have taken the 1998 machine 20 years to finish – it would be half done today! If it were possible for cars to improve their gas mileage over the past decade at the same rate that supercomputers have improved their cost and efficiency, we'd be getting 200,000 miles to the gallon today.

Supercomputer breaks petaflop barrier
Military machine smashes speed record

**********************

A US military supercomputer has broken the petaflop speed record by performing one thousand trillion calculations per second.

The 'Roadrunner' machine was developed at the Los Alamos Laboratory and uses AMD Opteron chips and the Cell processors found in PlayStation games consoles.

It will be used to model climate systems before being classified by the Army to work on nuclear explosion modelling.

"We replace our high-performance supercomputers every four or five years," said Andy White, leader of supercomputer development at Los Alamos.

"They become outdated in terms of speed, and the maintenance costs and failure rates get too high."

White explained that the use of two kinds of chips represents a revolution in supercomputing.

'Roadrunner' uses fewer than 20,000 processors while the next fastest computer, IBM's Blue Gene/L, uses over 200,000 and runs at half the speed.

The Los Alamos machine uses 7,000 multi-core Opteron processors and parses the data to 12,960 enhanced Cell processors using 57 miles of fibre-optic cabling. The device consumes three megawatts of power, enough to run 1,000 homes.

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.