Linux versus Mac OS X and Windows XP on Intel Dual Core

Berkeley ^ | Unknown | Jasjeet Sekhon

[ShadowAce]

Summary

Linux is found to be much faster than Apple's OS X for statistical computing. And although Linux is 5 to 10 percent faster than Windows XP, both are markedly faster than OS X. For example, in one benchmark both Linux and Windows XP are more than twice as fast as OS X. The results on this page were conducted on a MacBookpro with a 2.16Ghz Intel Core Duo chip and 2GB of RAM.

I had previous conducted Linux vs. Mac OS X and Windows XP and Opteron vs. G5 and Pentium benchmarks. Those results were terrible for OS X and not particularly good for the G5 (970) chip. For example, my 2.7 pound Pentium-M Linux laptop is faster than my 44 pound G5 running OS X. The floating point performance of the 970 chip leaves much to be desired, but OS X makes the performance problem significantly worse.

Given these results, this spoof of Apple's marketing campaign on a website which helps users install Linux is fitting:

The Intel chip... For months, it's been trapped inside a Mac, inside a pretty little box, dutifully performing pretty little tasks when it could have been doing so much more. Starting today, the Intel chip will be set free, and get to live life in a Mac... running Linux. Imagine the possibilities.

Background

People often ask me about my opinion of Apple's OS X both as an alternative to Linux and as an operating system useful for statistical computing. Because I support my statistical software on various platforms, I have to think about the idiosyncrasies of various operating systems and chips. In order to save time repeating the same information to many people, I have decided to post it on the web. The short answer: use Linux if you want performance and stability. If you want to use Mac OS X or Windows XP, go ahead. All of these operating systems are now above the line (not long ago the operating systems out of Redmond and Cupertino were a joke). However, if you decide to use Mac OS X for whatever reason, don't assume that it is just like Linux or some other efficient unix but with a friendly GUI. Life is full of tradeoffs and reasonable people can decide to make different choices. Don't pretend that tradeoffs don't exist, and don't fall victim to Apple's marketing which is an extension of the Steve Jobs Reality Distortion Field.

Benchmarks

I present here a set of benchmarks which are relevant to my work and to people working in statistical computing, particular people using the R Project for Statistical Computing. These benchmarks are floating point bound where the main IO is to memory and not to disk. Cache and Translation Look-aside Buffer (TLB) misses really matter as well as memory speed. This setup may be of more general interest. But they may not be relevant for what you do. If you need a computer to do Y, and these benchmarks are in no way related to Y, don't write me to complain about it. These benchmarks are useful for the work I and some other computational statistics people do.

OS X is incredibly slow by design in part because of the hybrid XNU kernel it uses. It is based on the Mach Microkernel (see Linus vs. Tanenbaum) and the excellent Berkeley Standard Distribution (BSD) kernel. The hybrid kernel is very inefficient and less stable than alternatives such as the Linux kernel and the BSD kernel found in FreeBSD. The reasons for this are many. For example, in Linux, the variables for a system call are passed directly using the register file. In OS X, they are packed up in a memory buffer, passed to a variety of places, and the results are then passed back using another memory buffer before the results are written back to the register file. You can just imagine what that does for TLB and cache hits. This just adds to the context switching difficulties on some chips such as the Power4. Memory management in OS X is awful. To quote Kazushige Goto talking about his BLAS: "Performance is suppressed on purpose due to [the] awful memory management of OS X". Goto's work is described and praised on Apple's own website because he added a custom BLAS for the Apple super computer at Virginia Tech. On the Apple site it states that Goto was "pulling out incredible efficiencies". Given Goto's own benchmarks and comments, it is not surprising that the Virginia Tech team was able to pull out these efficiencies by writing a kernel level memory manager to work around OS X's memory manager (information provided via email by their Director of HPC and Technology Innovation). The custom kernel level memory manager is not mentioned on Apple's webpage so the impression is left that the excellent performance achieved by the Virginia Tech team is possible with the standard operating system. This is just another example of the Steve Jobs Reality Distortion Field.

The benchmarks presented here are based on two of my statistical software packages for R: Matching (Multivariate and Propensity Score Matching Software) and rgenoud (R Version of GENetic Optimization Using Derivatives). The code uses C++ code extensively. The two benchmark scripts are available here (Genetic Matching) and here (Matching). All benchmarks were done using R-2.3 and gcc 4. The best timing result of the three calls to GenMatch in the GenMatch script are presented and the best result of three consecutive runs of the matching script are presented (examining the worst or the average times yields the same substantive results).

The machines are:

Label	OS and Chip
OS X Core Duo	Tiger on MacBookpro, Intel 2.16GHz Dual Core 2GB RAM
Linux Core Duo	Ubuntu Linux LiveCD on MacBookpro, Intel 2.16GHz Dual Core 2GB RAM. Note: Xorg server running with GNOME
XP Core Duo	Windows XP SP2 on MacBookpro, Intel 2.16GHz Dual Core 2GB RAM.
Linux P4	Ubuntu Linux (Drapper Drake Beta 2) on 3GHz Pentium 4, 2GB RAM. Note: Xgl+compiz running with KDE
Linux Opteron	Ubuntu Linux (64bit) on Opteron 250, 4GB RAM Note: Xorg server running with KDE

Both Linux and Windows XP are vastly faster than OS X: more than twice as fast. And Linux is somewhat faster than Windows XP. This benchmark does not take up much RAM, less than 30meg, nor does it work the filesystem much. But the application does flip between various shared libraries and pass various data objects back and forth in RAM. The following benchmark takes about the same amount of RAM, but unlike the previous one it does not flip between various shared libraries. It does call a shared library, but it only does it once and only passes results back once.



This second benchmarks looks better for OS X, but it is still about 1.2 times slower than Linux. And the gap between Linux and Windows has grown from about 5 to about 10 percent.

These benchmarks do not use a graphical user interface. They are batch jobs run from the command line and produce no graphical output. No X11 or Aqua calls are made. And on all platforms the benchmark process obtains 99%+ of a cpu or core. Moreover, in neither benchmark are we testing IO or running multiple processes on the same chip. If we do either of these, the Opteron's relative performance improves.

Many people commented that my previous benchmarks, which compared OS X on the G5 with Linux on Opteron chips, were limited because gcc is optimized for the x86 family. In these benchmarks, this excuse can obvious not be used. There are some serious issues with OS X and the gang in Cupertino should get to work. Even Windows XP performs better than OS X, which is embarrassing for Apple and a testament to Microsoft engineering given the legacy software requirements of Windows.

As noted before, the hybrid XNU kernel is probably to blame for OS X's problems. People on the web have recently been speculating whether Apple with drop the Mach micro-kernel portion of XNU. These rumors have picked with the departure of Avie Tevanian, an important figure in the development of the Mach Kernel first at Carnegie Mellon and then at Apple. Interestingly, Chris Emura, the Filesystem Development Manager within Apple's CoreOS organization, recently stated that Apple is interested in porting Sun's ZFS filesystem to OS X. If true, it may be that Apple is interested in fixing core issues with their operating system now that the eye candy is stable.

I have conducted many more benchmarks on these and other machines. For example, I have tested the HFS+ filesystem. It is slower than reiser especially for small and medium sized files and slower than XFS especially for large files. If you want these additional benchmarks, let me know.

There are claims on the web that when Apple developers compile OS X on the 970, they use -Os. That is, they optimize for size and not for performance. "So even though Apple talked a lot of smack about having a first-class 64-bit RISC workstation chip under the hood of their towers, in the end they were more concerned about OS X's bulging memory requirements than they were about The Snappy(TM)." AnandTech has an article which offers another explanation for why OS X is so inefficient. See No more mysteries: Apple's G5 versus x86, Mac OS X versus Linux.

A writeup of my previous benchmarks, which includes a review of my general impressions of OS X, is available HERE.

Suggestions

If you have any suggestions on how to fix the terrible performance of (this software on) OS X or if you think something here is erroneous, please contact me.

Links

See similar benchmarks available on AnandTech's website: "No more mysteries: Apple's G5 versus x86, Mac OS X versus Linux" and "No more mysteries, part two".

For another review see When a Linux user buys Apple's Mac mini.

[Eddie01]

Niche market pays higher premium to cover OS development costs.

Niche market shrinks.

Revenues fall.

Stock tumbles.

Investors bail.

How long till the whole thing blows up. Don't know but MS has a plan they're not telling us about, be sure of that.

MS doesn't want Apple to create a machine that runs both XP and OS X (Intel inside). It creates a swith over bridge. MS will do everything to make life miserable for Apple and will leverage its deeeeeep pockets to crush OS X.

Apple claims the core duo and solo switch was for integer and FP math benchmarked superiority, heat and energy savings.

I think the 14,000 prize money they offered to any hacker that could get XP to run on their machines is more telling. They need market share to sustain mounting development costs. That's the dirty little secret.

[Eddie01]

Is solaris compatible with the industry business standard MS Office Suite? i.e. can you share files between XP for example and Solaris OS?

If not you've got trouble with a capital "T".

[Golden Eagle]

Sun has a much closer partnership with Microsoft than any Linux company, which isn't about to change. As does Apple. The combination of OSX desktop on top of Solaris would be a powerful combination for both companies, one I believe is more likely then either switching completely to Linux. There's just too many legal questions about Linux, plus that leftist Stallman always trying to block GPL software from being compatible with DRM.

[zeugma]

Darn. I guess there's really now just one choice for all the millions and millions of people that just run statistics software.

Best response on thread. 

I'm not too terribly suprised that Macs are slower than Linux, but am sorta suprised that even MS-Windows beats OSX on this test. Then again, the 'benchmark' is, from the sound of it, not a good real-world test of abilities. 

I wonder how Linux/OSX/MS-Windows compare when folding?

[HAL9000]

Niche market pays higher premium to cover OS development costs.

Don't worry about Apple's development costs. Apple gets a lot more bang-for-the-buck than Microsoft.

How long till the whole thing blows up.

There are no signs that Apple will blow up in the foreseeable future. They are well-positioned for a decade of success.

I think the 14,000 prize money they offered to any hacker that could get XP to run on their machines is more telling. They need market share to sustain mounting development costs. That's the dirty little secret.

Apple did not sponsor that contest or contribute to the prize. It was strictly a user-community effort.

[Swordmaker]

XP is amortized over 130 million users, OS X over 3 million.

Your figures are invalid... Apple sold over 6 million OS X Macs just this last year alone... and studies indicate there are approximately 25 million OS X Macs in the installed base.

[Swordmaker]

interesting article PING...

If you want on or off the Mac Ping List, Freepmail me.

[Swordmaker]

Apple PC business will die unless they can convert en mass Win/Intel folks over to their Mac mini through iPod halo sales. Otherwise the next OS premium will drive the cost up so high the "niche" market will begin to erode.

This is FUD... and more of the Apple will die FUD that has been spread for the past 20 years. Apple is in better financial and market shape than it as been in years.

[wagglebee]

Well, I don't even know what "statistical" computing is. I just love the fact that my Mac doesn't have any problems compared to the PCs that I had cursed on a daily basis for years.

[Swordmaker]

...saw its U.S. market share decline from 3.6 percent to 3.5 percent.

... in a quarter in which buyers were waiting for the new Intel Macs to be released.

[Swordmaker]

I think the 14,000 prize money they offered to any hacker that could get XP to run on their machines is more telling. They need market share to sustain mounting development costs. That's the dirty little secret.

The "dirty little secret", Eddie, is that you don't know what you are talking about.

The $14,000 prize was a private fund, not associated with Apple, which was accumulated by donation from enthusiasts and hackers to award to the first private individuals who could boot XP on an MacIntel computer. One week after it was awarded, again privately, Apple released their own Boot Camp software that allows XP to be booted on any Intel based Mac that has nothing to do with the award.

[ThinkDifferent]

It cost roughly 1 billion to develop OS X and XP each.

Source? From what I've read OS X is much cheaper to develop than Windows, due to fewer hardware configurations to support and a cleaner code base.

As a result Apple will see its niche market evaporate

That's a common prediction which has been popular since 1984.

[Golden Eagle]

Well, I don't even know what "statistical" computing is. I just love the fact that my Mac doesn't have any problems compared to the PCs that I had cursed on a daily basis for years.

Don't sweat it. This is just another perfect example of the Linux pushers claiming their foreign clone of an O/S is the greatest technological feat of all human history.

[FLAMING DEATH]

Me too now. Thanks for the recommendation. Got it running on my laptop, and everything's a go...wireless, sound, video, and all the monitors for battery, CPU temp, etc. are working great.

Rock solid, stable, and fast (did you notice the big jump in boot speed from 3.4 to 6?). Can't believe it's still in beta, and can't wait until summer when the final comes out!

[quidnunc]

I am starting to sour on Apple — big time!

More and more audio streams are beoming inaccessible to my Mac.

The final straw was when my G5 wouldn't play my favorite audio stream from KRLA.

http://www2.krla870.com/listen/

What the hell good is a computer that isn't good for much of anything anything but purchasing music from the Apple store and transferring it to an iPod?

[Echo Talon]

I never ran MEPIS 3.4 that much... so im not sure about the boot times. I think its a great OS! :) glad that it's working for you! When the Final Kununtu comes out im gonna give that a shot. the Kubuntu Flight 7 was just released the other day.

Also if you have your laptop dual booting, and you want windows to be the default in the grub menu its easy just change the order in "menu.lst" file located in /boot/grub make it look something like this

timeout 15
color cyan/blue white/blue
foreground ffffff
background 0639a1

gfxmenu /boot/grub/message

title Windows XP Pro at sda1
rootnoverify (hd0,0)
chainloader +1

title MEPIS Linux at sda3, kernel 2.6.15-22-386
kernel /boot/vmlinuz-2.6.15-22-386 root=/dev/sda3 nomce quiet vga=791

title MEMTEST
kernel /boot/memtest86+.bin

[Echo Talon]

The final straw was when my G5 wouldn't play my favorite audio stream from KRLA.

I don't seem to be having ANY trouble streaming that radio station with MEPIS 6 Linux and Kaffeine....

[HAL9000]

It's a problem with the KRLA web page, but here is a workaround -

Open Windows Media Player

Select the "Open URL..." command under the file menu

Paste this URL into the box -

http://boss.streamos.com/wmedia-live/kkla/7723/16_kkla-16_krla_0105_050106.asx

then click the OK button. You should receive KRLA live. You can add it to Favorites for a future shortcut.

[Echo Talon]

wow, doing that opens Kaffeine player in a hurry. :)

[Echo Talon]

I don't think its a problem with the website, it works fine on Linux and Windows. Maybe its something funky in the MP software?