Linux versus Mac OS X and Windows XP on Intel Dual Core

Berkeley ^ | Unknown | Jasjeet Sekhon

[ShadowAce]

Summary

Linux is found to be much faster than Apple's OS X for statistical computing. And although Linux is 5 to 10 percent faster than Windows XP, both are markedly faster than OS X. For example, in one benchmark both Linux and Windows XP are more than twice as fast as OS X. The results on this page were conducted on a MacBookpro with a 2.16Ghz Intel Core Duo chip and 2GB of RAM.

I had previous conducted Linux vs. Mac OS X and Windows XP and Opteron vs. G5 and Pentium benchmarks. Those results were terrible for OS X and not particularly good for the G5 (970) chip. For example, my 2.7 pound Pentium-M Linux laptop is faster than my 44 pound G5 running OS X. The floating point performance of the 970 chip leaves much to be desired, but OS X makes the performance problem significantly worse.

Given these results, this spoof of Apple's marketing campaign on a website which helps users install Linux is fitting:

The Intel chip... For months, it's been trapped inside a Mac, inside a pretty little box, dutifully performing pretty little tasks when it could have been doing so much more. Starting today, the Intel chip will be set free, and get to live life in a Mac... running Linux. Imagine the possibilities.

Background

People often ask me about my opinion of Apple's OS X both as an alternative to Linux and as an operating system useful for statistical computing. Because I support my statistical software on various platforms, I have to think about the idiosyncrasies of various operating systems and chips. In order to save time repeating the same information to many people, I have decided to post it on the web. The short answer: use Linux if you want performance and stability. If you want to use Mac OS X or Windows XP, go ahead. All of these operating systems are now above the line (not long ago the operating systems out of Redmond and Cupertino were a joke). However, if you decide to use Mac OS X for whatever reason, don't assume that it is just like Linux or some other efficient unix but with a friendly GUI. Life is full of tradeoffs and reasonable people can decide to make different choices. Don't pretend that tradeoffs don't exist, and don't fall victim to Apple's marketing which is an extension of the Steve Jobs Reality Distortion Field.

Benchmarks

I present here a set of benchmarks which are relevant to my work and to people working in statistical computing, particular people using the R Project for Statistical Computing. These benchmarks are floating point bound where the main IO is to memory and not to disk. Cache and Translation Look-aside Buffer (TLB) misses really matter as well as memory speed. This setup may be of more general interest. But they may not be relevant for what you do. If you need a computer to do Y, and these benchmarks are in no way related to Y, don't write me to complain about it. These benchmarks are useful for the work I and some other computational statistics people do.

OS X is incredibly slow by design in part because of the hybrid XNU kernel it uses. It is based on the Mach Microkernel (see Linus vs. Tanenbaum) and the excellent Berkeley Standard Distribution (BSD) kernel. The hybrid kernel is very inefficient and less stable than alternatives such as the Linux kernel and the BSD kernel found in FreeBSD. The reasons for this are many. For example, in Linux, the variables for a system call are passed directly using the register file. In OS X, they are packed up in a memory buffer, passed to a variety of places, and the results are then passed back using another memory buffer before the results are written back to the register file. You can just imagine what that does for TLB and cache hits. This just adds to the context switching difficulties on some chips such as the Power4. Memory management in OS X is awful. To quote Kazushige Goto talking about his BLAS: "Performance is suppressed on purpose due to [the] awful memory management of OS X". Goto's work is described and praised on Apple's own website because he added a custom BLAS for the Apple super computer at Virginia Tech. On the Apple site it states that Goto was "pulling out incredible efficiencies". Given Goto's own benchmarks and comments, it is not surprising that the Virginia Tech team was able to pull out these efficiencies by writing a kernel level memory manager to work around OS X's memory manager (information provided via email by their Director of HPC and Technology Innovation). The custom kernel level memory manager is not mentioned on Apple's webpage so the impression is left that the excellent performance achieved by the Virginia Tech team is possible with the standard operating system. This is just another example of the Steve Jobs Reality Distortion Field.

The benchmarks presented here are based on two of my statistical software packages for R: Matching (Multivariate and Propensity Score Matching Software) and rgenoud (R Version of GENetic Optimization Using Derivatives). The code uses C++ code extensively. The two benchmark scripts are available here (Genetic Matching) and here (Matching). All benchmarks were done using R-2.3 and gcc 4. The best timing result of the three calls to GenMatch in the GenMatch script are presented and the best result of three consecutive runs of the matching script are presented (examining the worst or the average times yields the same substantive results).

The machines are:

Label	OS and Chip
OS X Core Duo	Tiger on MacBookpro, Intel 2.16GHz Dual Core 2GB RAM
Linux Core Duo	Ubuntu Linux LiveCD on MacBookpro, Intel 2.16GHz Dual Core 2GB RAM. Note: Xorg server running with GNOME
XP Core Duo	Windows XP SP2 on MacBookpro, Intel 2.16GHz Dual Core 2GB RAM.
Linux P4	Ubuntu Linux (Drapper Drake Beta 2) on 3GHz Pentium 4, 2GB RAM. Note: Xgl+compiz running with KDE
Linux Opteron	Ubuntu Linux (64bit) on Opteron 250, 4GB RAM Note: Xorg server running with KDE

Both Linux and Windows XP are vastly faster than OS X: more than twice as fast. And Linux is somewhat faster than Windows XP. This benchmark does not take up much RAM, less than 30meg, nor does it work the filesystem much. But the application does flip between various shared libraries and pass various data objects back and forth in RAM. The following benchmark takes about the same amount of RAM, but unlike the previous one it does not flip between various shared libraries. It does call a shared library, but it only does it once and only passes results back once.



This second benchmarks looks better for OS X, but it is still about 1.2 times slower than Linux. And the gap between Linux and Windows has grown from about 5 to about 10 percent.

These benchmarks do not use a graphical user interface. They are batch jobs run from the command line and produce no graphical output. No X11 or Aqua calls are made. And on all platforms the benchmark process obtains 99%+ of a cpu or core. Moreover, in neither benchmark are we testing IO or running multiple processes on the same chip. If we do either of these, the Opteron's relative performance improves.

Many people commented that my previous benchmarks, which compared OS X on the G5 with Linux on Opteron chips, were limited because gcc is optimized for the x86 family. In these benchmarks, this excuse can obvious not be used. There are some serious issues with OS X and the gang in Cupertino should get to work. Even Windows XP performs better than OS X, which is embarrassing for Apple and a testament to Microsoft engineering given the legacy software requirements of Windows.

As noted before, the hybrid XNU kernel is probably to blame for OS X's problems. People on the web have recently been speculating whether Apple with drop the Mach micro-kernel portion of XNU. These rumors have picked with the departure of Avie Tevanian, an important figure in the development of the Mach Kernel first at Carnegie Mellon and then at Apple. Interestingly, Chris Emura, the Filesystem Development Manager within Apple's CoreOS organization, recently stated that Apple is interested in porting Sun's ZFS filesystem to OS X. If true, it may be that Apple is interested in fixing core issues with their operating system now that the eye candy is stable.

I have conducted many more benchmarks on these and other machines. For example, I have tested the HFS+ filesystem. It is slower than reiser especially for small and medium sized files and slower than XFS especially for large files. If you want these additional benchmarks, let me know.

There are claims on the web that when Apple developers compile OS X on the 970, they use -Os. That is, they optimize for size and not for performance. "So even though Apple talked a lot of smack about having a first-class 64-bit RISC workstation chip under the hood of their towers, in the end they were more concerned about OS X's bulging memory requirements than they were about The Snappy(TM)." AnandTech has an article which offers another explanation for why OS X is so inefficient. See No more mysteries: Apple's G5 versus x86, Mac OS X versus Linux.

A writeup of my previous benchmarks, which includes a review of my general impressions of OS X, is available HERE.

Suggestions

If you have any suggestions on how to fix the terrible performance of (this software on) OS X or if you think something here is erroneous, please contact me.

Links

See similar benchmarks available on AnandTech's website: "No more mysteries: Apple's G5 versus x86, Mac OS X versus Linux" and "No more mysteries, part two".

For another review see When a Linux user buys Apple's Mac mini.

[ShadowAce]

...Linux pushers claiming their foreign clone of an O/S ...

Linux is just as American as Dulles Airport, the Gateway Arch, and quite a few others.

[FLAMING DEATH]

Looks good. I am a transparency freak.

I have transparent window titlebars, transparent taskbar, and semi-transparent menus all around.

This evening, I'll post you a screenshot of my Mepis 6 desktop.

[Golden Eagle]

So you can cobble together a few Chinese parts for less than it costs to buy a top of the line Mac. You think I'm impressed? LOL! I still wouldn't run Linux even if I was a cheapskate, you can run Solaris or BSD for free, and be using an original Unix instead of some foreign born clone with radical leftists like Richard Stallman behind it.

[antiRepublicrat]

How true. But, the main advocate for Mach was Avie Tevanian, and he has "left to spend more time with his family".

I have a feeling that the reason for going with Mach was to make Apple's "Just in Case" switch to Intel easier. We may even see Mach getting dumped in the future.

[antiRepublicrat]

huh? you have a problem with RAID?

I think he has a problem with RAID 0. I do, if the data is at all important.

[js1138]

A the moment the fastest consumer drives are the WD Raptors, which aren't very large by current standards. I've seen reviews that say some others are just as fast, but my benchmark is boot time and program loading time.

For sustained database access with multiuser or multi-threaded applications, I still think SCSI with a dedicated RAID controller is faster. I haven't done benchmarks on this, though.

If you encounter a drive failure in a business environment, nothing is faster than RAID 5 with hot swap. Running is faster than not running. I've been there.

[antiRepublicrat]

The economics are unsustainable for Apple.

Windows is an expensive monster. It is incredibly bloated with backwards compatibility, specific software compatibility patches, hardware compatibility, and an old architecture that its development costs are extremely high. Just look at how long Vista has taken, even with features ripped out. It's taken Microsoft longer to make Vista than it took Apple to make OS X, which was at a pretty stable 10.3 five years after its announcement.

Apple dumped their old OS rather than go this route. They used a free tried-and-true kernel with advanced OS features built on top. Their backwards compatibility was in one API separate from the others and a compatibility environment separate from the rest of the OS. It was a daring move with the potential for loss of user base, but it resulted in a much cleaner OS that costs less to maintain and advance.

BTW, estimates are about 20 million OS X users.

[N3WBI3]

You will only be able to run apple on the PPC until the cut over is complete, the apple license itself tells you what hardware you are allowed to run on thats not really choice.

As to Solaris yup I have finally decided to give them a second shot on the x86 platform after their stunt they pulled after 7 and they seem to have actually increased their driver support, nice to see..

Linux will run on at least the PPC, x86, SPARC, ARM, and MIPS processors not to mention all the embedded devices it will run on. By far with the exception of BSD Linux is the most versatile Operating system out there in terms of supported processors..

[N3WBI3]

Hal,

I don't use Safari all that much but cant you set up media player as a separate helper application which will pick up from that link?

[N3WBI3]

It cost roughly 1 billion to develop OS X and XP each. XP is amortized over 130 million users, OS X over 3 million. The next OS will cost between 1.5 and 2 million to develop. The economics are unsustainable for Apple.

BTW do you have a source for all of this?

[daviddennis]

I don't think these findings will make much difference, because people like using Apple computers. The human factors design is much better on the Mac and that's more important than a few points in performance. Also software availability is a huge advantage over Linux, and software UNavailability (of virii and spyware) is a huge advantage over Windows.

Some of the most innovative and highest quality software is coming out of Apple nowadays - think Final Cut Pro, Motion and Aperture. (Yes, Aperture has its problems, but the design is innovative and the problems are being worked out.)

That being said, I'm sure adjustments are being made to improve matters, and this may be why Avie is no longer with the company.

D

[Golden Eagle]

You will only be able to run apple on the PPC until the cut over is complete

Typical bunk from you, anyone with a PPC Mac can run it for as long as they like. What did you think, Apple was somehow going to stop anyone with a PPC Mac from even booting up? Probably, since they're evil capitalists LOL.

[ThinkDifferent]

I think he has a problem with RAID 0. I do, if the data is at all important.

Exactly. It shouldn't even be called RAID, because it's the exact opposite of "redundant". RAID 5 or 10 is good, but then you need more than 2 drives. And of course just mirroring is the way to go if reliability is critical and capacity and speed aren't.

[N3WBI3]

Oh I see youll have the choice of not buying a new system that runs on the ppc.. wow what a great consumer choice that is..

Talk about trying to start a fight over semantics..

[tortoise]

For sustained database access with multiuser or multi-threaded applications, I still think SCSI with a dedicated RAID controller is faster. I haven't done benchmarks on this, though.

SCSI drives are faster in an absolute sense, but with a good RAID control (LSI Logic being canonical) you can squeeze almost as much performance out of SATA arrays given identical disk parameters e.g. 10k SATA (which are repackaged SCSI) and 10k SCSI. SCSI has very mature I/O schedulers, and it frequently shows under parallel write loads (e.g. multi-user database systems). That said, a proper RAID controller with a decent sized writeback cache (and BBU please) will frequently hide enough of that I/O scheduling that the performance is very similar most of the time. For simple mirroring, soft RAID often works as well as hardware RAID.

RAID levels 1,5,6, and 10 are your best bets depending on the situation. For big arrays, 6 and 10. RAID 5 has excellent read performance and is stingy with disk space, but it will become nigh unusable for big arrays when operating in degraded mode due to a disk failure, hence RAID 6. RAID 6 is slighty less stingy (it requires two parity disks instead of one) and has most of the read performance (one less spindle), but performance will not degrade in the case of single disk failure and it provides protection against the rarer but still too common double disk failure. Unfortunately, not that many RAID controllers support RAID 6.

[unixfox]

Considering he only has 2 drives installed, his best bet is Raid 1.

[js1138]

I deal with small busnesses that take orders over the phone. They don't nedd a lot of performance, but they need 100 percent uptime. A raid 5 failure just means a bit of a slowdown.

They now have hotspares installed, so I can take my time replacing a dead drive. I've only had to deal with one drive failure on a production server.

[antiRepublicrat]

Exactly. It shouldn't even be called RAID, because it's the exact opposite of "redundant".

It wasn't even in the original definition of RAID. They stuck level 0 before the original levels 1-5. Unless I need a really fast system for temporary data (e.g., writing uncompressed 1080p video before it's moved off to permanent storage), I'd rather go JBOD instead of 0.

[FLAMING DEATH]

I always edit the line that says, "Windows XP" to say, "Billy Bob's Super Swell Operatin' System".

That always cracks me up.

[ShadowAce]

...its easy just change the order in "menu.lst" file located in /boot/grub make it look something like this...

It would be easier to just add a default line in your grub.conf file.

timeout 15
color cyan/blue white/blue
foreground ffffff
background 0639a1
default=2
gfxmenu /boot/grub/message

The numbering for the default line is 0-based.