Linux versus Mac OS X and Windows XP on Intel Dual Core

Berkeley ^ | Unknown | Jasjeet Sekhon

[ShadowAce]

Summary

Linux is found to be much faster than Apple's OS X for statistical computing. And although Linux is 5 to 10 percent faster than Windows XP, both are markedly faster than OS X. For example, in one benchmark both Linux and Windows XP are more than twice as fast as OS X. The results on this page were conducted on a MacBookpro with a 2.16Ghz Intel Core Duo chip and 2GB of RAM.

I had previous conducted Linux vs. Mac OS X and Windows XP and Opteron vs. G5 and Pentium benchmarks. Those results were terrible for OS X and not particularly good for the G5 (970) chip. For example, my 2.7 pound Pentium-M Linux laptop is faster than my 44 pound G5 running OS X. The floating point performance of the 970 chip leaves much to be desired, but OS X makes the performance problem significantly worse.

Given these results, this spoof of Apple's marketing campaign on a website which helps users install Linux is fitting:

The Intel chip... For months, it's been trapped inside a Mac, inside a pretty little box, dutifully performing pretty little tasks when it could have been doing so much more. Starting today, the Intel chip will be set free, and get to live life in a Mac... running Linux. Imagine the possibilities.

Background

People often ask me about my opinion of Apple's OS X both as an alternative to Linux and as an operating system useful for statistical computing. Because I support my statistical software on various platforms, I have to think about the idiosyncrasies of various operating systems and chips. In order to save time repeating the same information to many people, I have decided to post it on the web. The short answer: use Linux if you want performance and stability. If you want to use Mac OS X or Windows XP, go ahead. All of these operating systems are now above the line (not long ago the operating systems out of Redmond and Cupertino were a joke). However, if you decide to use Mac OS X for whatever reason, don't assume that it is just like Linux or some other efficient unix but with a friendly GUI. Life is full of tradeoffs and reasonable people can decide to make different choices. Don't pretend that tradeoffs don't exist, and don't fall victim to Apple's marketing which is an extension of the Steve Jobs Reality Distortion Field.

Benchmarks

I present here a set of benchmarks which are relevant to my work and to people working in statistical computing, particular people using the R Project for Statistical Computing. These benchmarks are floating point bound where the main IO is to memory and not to disk. Cache and Translation Look-aside Buffer (TLB) misses really matter as well as memory speed. This setup may be of more general interest. But they may not be relevant for what you do. If you need a computer to do Y, and these benchmarks are in no way related to Y, don't write me to complain about it. These benchmarks are useful for the work I and some other computational statistics people do.

OS X is incredibly slow by design in part because of the hybrid XNU kernel it uses. It is based on the Mach Microkernel (see Linus vs. Tanenbaum) and the excellent Berkeley Standard Distribution (BSD) kernel. The hybrid kernel is very inefficient and less stable than alternatives such as the Linux kernel and the BSD kernel found in FreeBSD. The reasons for this are many. For example, in Linux, the variables for a system call are passed directly using the register file. In OS X, they are packed up in a memory buffer, passed to a variety of places, and the results are then passed back using another memory buffer before the results are written back to the register file. You can just imagine what that does for TLB and cache hits. This just adds to the context switching difficulties on some chips such as the Power4. Memory management in OS X is awful. To quote Kazushige Goto talking about his BLAS: "Performance is suppressed on purpose due to [the] awful memory management of OS X". Goto's work is described and praised on Apple's own website because he added a custom BLAS for the Apple super computer at Virginia Tech. On the Apple site it states that Goto was "pulling out incredible efficiencies". Given Goto's own benchmarks and comments, it is not surprising that the Virginia Tech team was able to pull out these efficiencies by writing a kernel level memory manager to work around OS X's memory manager (information provided via email by their Director of HPC and Technology Innovation). The custom kernel level memory manager is not mentioned on Apple's webpage so the impression is left that the excellent performance achieved by the Virginia Tech team is possible with the standard operating system. This is just another example of the Steve Jobs Reality Distortion Field.

The benchmarks presented here are based on two of my statistical software packages for R: Matching (Multivariate and Propensity Score Matching Software) and rgenoud (R Version of GENetic Optimization Using Derivatives). The code uses C++ code extensively. The two benchmark scripts are available here (Genetic Matching) and here (Matching). All benchmarks were done using R-2.3 and gcc 4. The best timing result of the three calls to GenMatch in the GenMatch script are presented and the best result of three consecutive runs of the matching script are presented (examining the worst or the average times yields the same substantive results).

The machines are:

Label	OS and Chip
OS X Core Duo	Tiger on MacBookpro, Intel 2.16GHz Dual Core 2GB RAM
Linux Core Duo	Ubuntu Linux LiveCD on MacBookpro, Intel 2.16GHz Dual Core 2GB RAM. Note: Xorg server running with GNOME
XP Core Duo	Windows XP SP2 on MacBookpro, Intel 2.16GHz Dual Core 2GB RAM.
Linux P4	Ubuntu Linux (Drapper Drake Beta 2) on 3GHz Pentium 4, 2GB RAM. Note: Xgl+compiz running with KDE
Linux Opteron	Ubuntu Linux (64bit) on Opteron 250, 4GB RAM Note: Xorg server running with KDE

Both Linux and Windows XP are vastly faster than OS X: more than twice as fast. And Linux is somewhat faster than Windows XP. This benchmark does not take up much RAM, less than 30meg, nor does it work the filesystem much. But the application does flip between various shared libraries and pass various data objects back and forth in RAM. The following benchmark takes about the same amount of RAM, but unlike the previous one it does not flip between various shared libraries. It does call a shared library, but it only does it once and only passes results back once.



This second benchmarks looks better for OS X, but it is still about 1.2 times slower than Linux. And the gap between Linux and Windows has grown from about 5 to about 10 percent.

These benchmarks do not use a graphical user interface. They are batch jobs run from the command line and produce no graphical output. No X11 or Aqua calls are made. And on all platforms the benchmark process obtains 99%+ of a cpu or core. Moreover, in neither benchmark are we testing IO or running multiple processes on the same chip. If we do either of these, the Opteron's relative performance improves.

Many people commented that my previous benchmarks, which compared OS X on the G5 with Linux on Opteron chips, were limited because gcc is optimized for the x86 family. In these benchmarks, this excuse can obvious not be used. There are some serious issues with OS X and the gang in Cupertino should get to work. Even Windows XP performs better than OS X, which is embarrassing for Apple and a testament to Microsoft engineering given the legacy software requirements of Windows.

As noted before, the hybrid XNU kernel is probably to blame for OS X's problems. People on the web have recently been speculating whether Apple with drop the Mach micro-kernel portion of XNU. These rumors have picked with the departure of Avie Tevanian, an important figure in the development of the Mach Kernel first at Carnegie Mellon and then at Apple. Interestingly, Chris Emura, the Filesystem Development Manager within Apple's CoreOS organization, recently stated that Apple is interested in porting Sun's ZFS filesystem to OS X. If true, it may be that Apple is interested in fixing core issues with their operating system now that the eye candy is stable.

I have conducted many more benchmarks on these and other machines. For example, I have tested the HFS+ filesystem. It is slower than reiser especially for small and medium sized files and slower than XFS especially for large files. If you want these additional benchmarks, let me know.

There are claims on the web that when Apple developers compile OS X on the 970, they use -Os. That is, they optimize for size and not for performance. "So even though Apple talked a lot of smack about having a first-class 64-bit RISC workstation chip under the hood of their towers, in the end they were more concerned about OS X's bulging memory requirements than they were about The Snappy(TM)." AnandTech has an article which offers another explanation for why OS X is so inefficient. See No more mysteries: Apple's G5 versus x86, Mac OS X versus Linux.

A writeup of my previous benchmarks, which includes a review of my general impressions of OS X, is available HERE.

Suggestions

If you have any suggestions on how to fix the terrible performance of (this software on) OS X or if you think something here is erroneous, please contact me.

Links

See similar benchmarks available on AnandTech's website: "No more mysteries: Apple's G5 versus x86, Mac OS X versus Linux" and "No more mysteries, part two".

For another review see When a Linux user buys Apple's Mac mini.

[antiRepublicrat]

I stated no modem, no wireless, no server stuff just to show you that I wasn't jacking up the price

What's necessary depends on your needs. In comparison, I'd suggest comparing apples to apples. That's kind of hard with the PPC-based Macs, which are generally bought for professional use, not the area in which you're comparing.