Linux versus Mac OS X and Windows XP on Intel Dual Core

Berkeley ^ | Unknown | Jasjeet Sekhon

[ShadowAce]

Summary

Linux is found to be much faster than Apple's OS X for statistical computing. And although Linux is 5 to 10 percent faster than Windows XP, both are markedly faster than OS X. For example, in one benchmark both Linux and Windows XP are more than twice as fast as OS X. The results on this page were conducted on a MacBookpro with a 2.16Ghz Intel Core Duo chip and 2GB of RAM.

I had previous conducted Linux vs. Mac OS X and Windows XP and Opteron vs. G5 and Pentium benchmarks. Those results were terrible for OS X and not particularly good for the G5 (970) chip. For example, my 2.7 pound Pentium-M Linux laptop is faster than my 44 pound G5 running OS X. The floating point performance of the 970 chip leaves much to be desired, but OS X makes the performance problem significantly worse.

Given these results, this spoof of Apple's marketing campaign on a website which helps users install Linux is fitting:

The Intel chip... For months, it's been trapped inside a Mac, inside a pretty little box, dutifully performing pretty little tasks when it could have been doing so much more. Starting today, the Intel chip will be set free, and get to live life in a Mac... running Linux. Imagine the possibilities.

Background

People often ask me about my opinion of Apple's OS X both as an alternative to Linux and as an operating system useful for statistical computing. Because I support my statistical software on various platforms, I have to think about the idiosyncrasies of various operating systems and chips. In order to save time repeating the same information to many people, I have decided to post it on the web. The short answer: use Linux if you want performance and stability. If you want to use Mac OS X or Windows XP, go ahead. All of these operating systems are now above the line (not long ago the operating systems out of Redmond and Cupertino were a joke). However, if you decide to use Mac OS X for whatever reason, don't assume that it is just like Linux or some other efficient unix but with a friendly GUI. Life is full of tradeoffs and reasonable people can decide to make different choices. Don't pretend that tradeoffs don't exist, and don't fall victim to Apple's marketing which is an extension of the Steve Jobs Reality Distortion Field.

Benchmarks

I present here a set of benchmarks which are relevant to my work and to people working in statistical computing, particular people using the R Project for Statistical Computing. These benchmarks are floating point bound where the main IO is to memory and not to disk. Cache and Translation Look-aside Buffer (TLB) misses really matter as well as memory speed. This setup may be of more general interest. But they may not be relevant for what you do. If you need a computer to do Y, and these benchmarks are in no way related to Y, don't write me to complain about it. These benchmarks are useful for the work I and some other computational statistics people do.

OS X is incredibly slow by design in part because of the hybrid XNU kernel it uses. It is based on the Mach Microkernel (see Linus vs. Tanenbaum) and the excellent Berkeley Standard Distribution (BSD) kernel. The hybrid kernel is very inefficient and less stable than alternatives such as the Linux kernel and the BSD kernel found in FreeBSD. The reasons for this are many. For example, in Linux, the variables for a system call are passed directly using the register file. In OS X, they are packed up in a memory buffer, passed to a variety of places, and the results are then passed back using another memory buffer before the results are written back to the register file. You can just imagine what that does for TLB and cache hits. This just adds to the context switching difficulties on some chips such as the Power4. Memory management in OS X is awful. To quote Kazushige Goto talking about his BLAS: "Performance is suppressed on purpose due to [the] awful memory management of OS X". Goto's work is described and praised on Apple's own website because he added a custom BLAS for the Apple super computer at Virginia Tech. On the Apple site it states that Goto was "pulling out incredible efficiencies". Given Goto's own benchmarks and comments, it is not surprising that the Virginia Tech team was able to pull out these efficiencies by writing a kernel level memory manager to work around OS X's memory manager (information provided via email by their Director of HPC and Technology Innovation). The custom kernel level memory manager is not mentioned on Apple's webpage so the impression is left that the excellent performance achieved by the Virginia Tech team is possible with the standard operating system. This is just another example of the Steve Jobs Reality Distortion Field.

The benchmarks presented here are based on two of my statistical software packages for R: Matching (Multivariate and Propensity Score Matching Software) and rgenoud (R Version of GENetic Optimization Using Derivatives). The code uses C++ code extensively. The two benchmark scripts are available here (Genetic Matching) and here (Matching). All benchmarks were done using R-2.3 and gcc 4. The best timing result of the three calls to GenMatch in the GenMatch script are presented and the best result of three consecutive runs of the matching script are presented (examining the worst or the average times yields the same substantive results).

The machines are:

Label	OS and Chip
OS X Core Duo	Tiger on MacBookpro, Intel 2.16GHz Dual Core 2GB RAM
Linux Core Duo	Ubuntu Linux LiveCD on MacBookpro, Intel 2.16GHz Dual Core 2GB RAM. Note: Xorg server running with GNOME
XP Core Duo	Windows XP SP2 on MacBookpro, Intel 2.16GHz Dual Core 2GB RAM.
Linux P4	Ubuntu Linux (Drapper Drake Beta 2) on 3GHz Pentium 4, 2GB RAM. Note: Xgl+compiz running with KDE
Linux Opteron	Ubuntu Linux (64bit) on Opteron 250, 4GB RAM Note: Xorg server running with KDE

Both Linux and Windows XP are vastly faster than OS X: more than twice as fast. And Linux is somewhat faster than Windows XP. This benchmark does not take up much RAM, less than 30meg, nor does it work the filesystem much. But the application does flip between various shared libraries and pass various data objects back and forth in RAM. The following benchmark takes about the same amount of RAM, but unlike the previous one it does not flip between various shared libraries. It does call a shared library, but it only does it once and only passes results back once.



This second benchmarks looks better for OS X, but it is still about 1.2 times slower than Linux. And the gap between Linux and Windows has grown from about 5 to about 10 percent.

These benchmarks do not use a graphical user interface. They are batch jobs run from the command line and produce no graphical output. No X11 or Aqua calls are made. And on all platforms the benchmark process obtains 99%+ of a cpu or core. Moreover, in neither benchmark are we testing IO or running multiple processes on the same chip. If we do either of these, the Opteron's relative performance improves.

Many people commented that my previous benchmarks, which compared OS X on the G5 with Linux on Opteron chips, were limited because gcc is optimized for the x86 family. In these benchmarks, this excuse can obvious not be used. There are some serious issues with OS X and the gang in Cupertino should get to work. Even Windows XP performs better than OS X, which is embarrassing for Apple and a testament to Microsoft engineering given the legacy software requirements of Windows.

As noted before, the hybrid XNU kernel is probably to blame for OS X's problems. People on the web have recently been speculating whether Apple with drop the Mach micro-kernel portion of XNU. These rumors have picked with the departure of Avie Tevanian, an important figure in the development of the Mach Kernel first at Carnegie Mellon and then at Apple. Interestingly, Chris Emura, the Filesystem Development Manager within Apple's CoreOS organization, recently stated that Apple is interested in porting Sun's ZFS filesystem to OS X. If true, it may be that Apple is interested in fixing core issues with their operating system now that the eye candy is stable.

I have conducted many more benchmarks on these and other machines. For example, I have tested the HFS+ filesystem. It is slower than reiser especially for small and medium sized files and slower than XFS especially for large files. If you want these additional benchmarks, let me know.

There are claims on the web that when Apple developers compile OS X on the 970, they use -Os. That is, they optimize for size and not for performance. "So even though Apple talked a lot of smack about having a first-class 64-bit RISC workstation chip under the hood of their towers, in the end they were more concerned about OS X's bulging memory requirements than they were about The Snappy(TM)." AnandTech has an article which offers another explanation for why OS X is so inefficient. See No more mysteries: Apple's G5 versus x86, Mac OS X versus Linux.

A writeup of my previous benchmarks, which includes a review of my general impressions of OS X, is available HERE.

Suggestions

If you have any suggestions on how to fix the terrible performance of (this software on) OS X or if you think something here is erroneous, please contact me.

Links

See similar benchmarks available on AnandTech's website: "No more mysteries: Apple's G5 versus x86, Mac OS X versus Linux" and "No more mysteries, part two".

For another review see When a Linux user buys Apple's Mac mini.

[ShadowAce]

I did my best in correcting the HTML for the links in this article. If the link you are trying does no work, go to the original article and follow it from there.

This article's HTML is not pretty.

[ShadowAce]

This is posted to provoke discussion on the methods involved and its findings. It is NOT meant to provoke a flame war!

[CodeToad]

All all fairness to each OS, some OSes do more than others under the covers and that can lead to performance differences.

[NYURepublican]

I didnt read the entire article but doesn't the fact that a majority of Mac OS programs are at a disadvantage over Linux and Win XP because the programs aren't really built for intel nor are they adapted yet for its architecture make these findings somewhat bogus. I had read on CNET that Mac OS X is considerably faster than Win XP on a Macbook Pro....i really dont care though just figured it should be mentioned.

[rarestia]

Benchmark tests are like opinions... everyone has one. The BIG2 graphics manufacturers (ATI & nVIDIA) have been wrestling for years on how to benchmark to make their product look better. My view on it is like weight loss: you may not see it on the scale, but how do you look in the mirror?

Take it for what it's worth: test them all, see which fits your needs best.

With dual core processing, you're going to have more power than you need for the typical home user. If you're running World of Warcraft, Oblivion, MS Outlook, MS Word and Firefox all at once, you may benefit from paying attention to the benchmarks. Otherwise, just shop within your means.

[Echo Talon]

I have MEPIS 6 beta 2 running on my Athlon64 3500+, seems to work pretty good... I'm thinking about downloading Flight 7 of Kubuntu X64 edition and give it a spin.

[glorgau]

OS X is incredibly slow by design in part because of the hybrid XNU kernel it uses. It is based on the Mach Microkernel (see Linus vs. Tanenbaum) and the excellent Berkeley Standard Distribution (BSD) kernel. The hybrid kernel is very inefficient and less stable than alternatives such as the Linux kernel and the BSD kernel found in FreeBSD.

How true. But, the main advocate for Mach was Avie Tevanian, and he has "left to spend more time with his family".

[MadIvan]

I have an HP Compaq nx7000, purchased in early 2004, using Ubuntu Linux Breezy Badger 5.10. My mother has a 15in Powerbook, purchased last year.

Based on the speeds I've noticed, I get considerably better performance out of the Linux laptop. I'd be interested to see the difference with MacBook Pro.

Regards, Ivan

[N3WBI3]

Linux is found to be much faster than Apple's OS X for statistical computing.

Couple of thoughts..

First: OSX is not really in the statistical computing market (yes yes I know there are super computers out there running it but truth be known enough nodes of any os make for a good supercomputer) so while this *might* be 100% correct it is by no means too bad a hit for apple. Now lets put it through the paces as a desktop OS running the same apps one might run on Linux and I think any disparity will disappear.

Secondly: OSX is still pretty new on Intel chips, lets see what the next OSX release does..

Thirdly: There is something to be said for portability and OSX was moved from one processor to another with little trouble. Any lack of statistical performance may be less a problem with design and more a reality of business / engineering decision in what should be focused on.

[avg_freeper]

Darn. I guess there's really now just one choice for all the millions and millions of people that just run statistics software.

[Eddie01]

It cost roughly 1 billion to develop OS X and XP each. XP is amortized over 130 million users, OS X over 3 million. The next OS will cost between 1.5 and 2 million to develop. The economics are unsustainable for Apple.

As a result Apple will see its niche market evaporate and is banking on the Mac mini to be the "hit" that causes mass swithing to its low cost Intel box (which runs XP using beta Bootcamp available with OS X 10.5 end of this year but downloadable now).

What Apple is banking on is the trend to home digital media and their Frontrow multimedia solution with Bonjour is quite impressive. It won't work though, not because of the poor benchmarks no real consumer with bucks and a cocain habit cares about, but because the Mac mini is a personal space device without personalization options and you have to spend another 5 - 10 grand on speakers and an HDTV to reap the real benefit.

Apple is facing a real crisis as its PC business accounts for the majority of its revenue even though they are selling tons more iPods.

[HAL9000]

Mac OS X is plenty fast for the average user, but for tasks that spawn a bunch of separate processes, like some database software, it is not as fast as Linux.

I believe Apple will improve the kernel's thread performance in the 10.5 release. It is possible that Apple could switch to a monolithic kernel, like Linux.

[ShadowAce]

It is possible that Apple could switch to a monolithic kernel, like Linux.

That would make it easier to maintain.

[N3WBI3]

Hmmm

MS has saturated the market has seen it browser and os start to lose share as OSX/Linux and Safari/Firefox begin to gain and Apple is in trouble?

Every release of OSX and every new apple product cycle increase not only the number of users but their relative market share to Microsoft.

You're also ignoring the fact Apple and MS are not even in the same market nobody buys a computer from Microsoft they do buy systems from Apple..

[Eddie01]

Every release of OSX and every new apple product cycle increase not only the number of users but their relative market share to Microsoft.

Globally?

You're also ignoring the fact Apple and MS are not even in the same market nobody buys a computer from Microsoft they do buy systems from Apple

No, quite the opposite. If you read what I wrote you will discover that the "niche" market Apple enjoys will not sustain OS X development costs in the future. They must appeal to the "mass" market which is where the "low cost" the Mac mini with Bootcamp and Rosetta (geeze I wonder why they needed those SW compatibility bridges??)idea came from.

Apple PC business will die unless they can convert en mass Win/Intel folks over to their Mac mini through iPod halo sales. Otherwise the next OS premium will drive the cost up so high the "niche" market will begin to erode.

[HAL9000]

It cost roughly 1 billion to develop OS X and XP each. XP is amortized over 130 million users, OS X over 3 million. The next OS will cost between 1.5 and 2 million to develop. The economics are unsustainable for Apple.

I disagree for several reasons - including, Apple has $8.23 billion cash in the bank.

[N3WBI3]

Globally?

Yes

If you read what I wrote you will discover that the "niche" market Apple enjoys will not sustain OS X development costs in the future.

If this were true apple would be losing money your logic is flawed in three ways.

First: Apple's market is growing at a rate faster than MS's their market was under 2% when OSX was first released it is now (depending on who you talk to) at somewhere between 4 and 5. This means the audience they are developing for is twice as large as when OSX was developed.

Second: Apple sells computers, MS sells operating systems all MS gets is the money from the people who buy their OS and applications, apple get considerably more.

Finally: Your cost estimation for apple versus XP ignored the fact Apple made an all out architecture change where as 2000 to XP was a minor kernel and library change. Going forward the cost of future apple os development will be considerably less..

[Golden Eagle]

Berkeley | Unknown | Jasjeet Sekhon

Yeah right whatever LOL

[Eddie01]

By Michael Kanellos, CNET News.com

Published on ZDNet News: April 19, 2006, 4:34 PM PT

http://news.zdnet.com/2100-9584_22-6062973.html Apple

...Computer so far hasn't been a boon to Intel. Apple, ranked fifth in the U.S. but not in the top five worldwide, saw its U.S. market share decline from 3.6 percent to 3.5 percent. Apple's worldwide share is around 2.3 percent.

[Golden Eagle]

Apple is more likely to switch to Solaris IMO. Did you see the announcement about ZFS?