Linux versus Mac OS X and Windows XP on Intel Dual Core

Berkeley ^ | Unknown | Jasjeet Sekhon

[ShadowAce]

Summary

Linux is found to be much faster than Apple's OS X for statistical computing. And although Linux is 5 to 10 percent faster than Windows XP, both are markedly faster than OS X. For example, in one benchmark both Linux and Windows XP are more than twice as fast as OS X. The results on this page were conducted on a MacBookpro with a 2.16Ghz Intel Core Duo chip and 2GB of RAM.

I had previous conducted Linux vs. Mac OS X and Windows XP and Opteron vs. G5 and Pentium benchmarks. Those results were terrible for OS X and not particularly good for the G5 (970) chip. For example, my 2.7 pound Pentium-M Linux laptop is faster than my 44 pound G5 running OS X. The floating point performance of the 970 chip leaves much to be desired, but OS X makes the performance problem significantly worse.

Given these results, this spoof of Apple's marketing campaign on a website which helps users install Linux is fitting:

The Intel chip... For months, it's been trapped inside a Mac, inside a pretty little box, dutifully performing pretty little tasks when it could have been doing so much more. Starting today, the Intel chip will be set free, and get to live life in a Mac... running Linux. Imagine the possibilities.

Background

People often ask me about my opinion of Apple's OS X both as an alternative to Linux and as an operating system useful for statistical computing. Because I support my statistical software on various platforms, I have to think about the idiosyncrasies of various operating systems and chips. In order to save time repeating the same information to many people, I have decided to post it on the web. The short answer: use Linux if you want performance and stability. If you want to use Mac OS X or Windows XP, go ahead. All of these operating systems are now above the line (not long ago the operating systems out of Redmond and Cupertino were a joke). However, if you decide to use Mac OS X for whatever reason, don't assume that it is just like Linux or some other efficient unix but with a friendly GUI. Life is full of tradeoffs and reasonable people can decide to make different choices. Don't pretend that tradeoffs don't exist, and don't fall victim to Apple's marketing which is an extension of the Steve Jobs Reality Distortion Field.

Benchmarks

I present here a set of benchmarks which are relevant to my work and to people working in statistical computing, particular people using the R Project for Statistical Computing. These benchmarks are floating point bound where the main IO is to memory and not to disk. Cache and Translation Look-aside Buffer (TLB) misses really matter as well as memory speed. This setup may be of more general interest. But they may not be relevant for what you do. If you need a computer to do Y, and these benchmarks are in no way related to Y, don't write me to complain about it. These benchmarks are useful for the work I and some other computational statistics people do.

OS X is incredibly slow by design in part because of the hybrid XNU kernel it uses. It is based on the Mach Microkernel (see Linus vs. Tanenbaum) and the excellent Berkeley Standard Distribution (BSD) kernel. The hybrid kernel is very inefficient and less stable than alternatives such as the Linux kernel and the BSD kernel found in FreeBSD. The reasons for this are many. For example, in Linux, the variables for a system call are passed directly using the register file. In OS X, they are packed up in a memory buffer, passed to a variety of places, and the results are then passed back using another memory buffer before the results are written back to the register file. You can just imagine what that does for TLB and cache hits. This just adds to the context switching difficulties on some chips such as the Power4. Memory management in OS X is awful. To quote Kazushige Goto talking about his BLAS: "Performance is suppressed on purpose due to [the] awful memory management of OS X". Goto's work is described and praised on Apple's own website because he added a custom BLAS for the Apple super computer at Virginia Tech. On the Apple site it states that Goto was "pulling out incredible efficiencies". Given Goto's own benchmarks and comments, it is not surprising that the Virginia Tech team was able to pull out these efficiencies by writing a kernel level memory manager to work around OS X's memory manager (information provided via email by their Director of HPC and Technology Innovation). The custom kernel level memory manager is not mentioned on Apple's webpage so the impression is left that the excellent performance achieved by the Virginia Tech team is possible with the standard operating system. This is just another example of the Steve Jobs Reality Distortion Field.

The benchmarks presented here are based on two of my statistical software packages for R: Matching (Multivariate and Propensity Score Matching Software) and rgenoud (R Version of GENetic Optimization Using Derivatives). The code uses C++ code extensively. The two benchmark scripts are available here (Genetic Matching) and here (Matching). All benchmarks were done using R-2.3 and gcc 4. The best timing result of the three calls to GenMatch in the GenMatch script are presented and the best result of three consecutive runs of the matching script are presented (examining the worst or the average times yields the same substantive results).

The machines are:

Label	OS and Chip
OS X Core Duo	Tiger on MacBookpro, Intel 2.16GHz Dual Core 2GB RAM
Linux Core Duo	Ubuntu Linux LiveCD on MacBookpro, Intel 2.16GHz Dual Core 2GB RAM. Note: Xorg server running with GNOME
XP Core Duo	Windows XP SP2 on MacBookpro, Intel 2.16GHz Dual Core 2GB RAM.
Linux P4	Ubuntu Linux (Drapper Drake Beta 2) on 3GHz Pentium 4, 2GB RAM. Note: Xgl+compiz running with KDE
Linux Opteron	Ubuntu Linux (64bit) on Opteron 250, 4GB RAM Note: Xorg server running with KDE

Both Linux and Windows XP are vastly faster than OS X: more than twice as fast. And Linux is somewhat faster than Windows XP. This benchmark does not take up much RAM, less than 30meg, nor does it work the filesystem much. But the application does flip between various shared libraries and pass various data objects back and forth in RAM. The following benchmark takes about the same amount of RAM, but unlike the previous one it does not flip between various shared libraries. It does call a shared library, but it only does it once and only passes results back once.



This second benchmarks looks better for OS X, but it is still about 1.2 times slower than Linux. And the gap between Linux and Windows has grown from about 5 to about 10 percent.

These benchmarks do not use a graphical user interface. They are batch jobs run from the command line and produce no graphical output. No X11 or Aqua calls are made. And on all platforms the benchmark process obtains 99%+ of a cpu or core. Moreover, in neither benchmark are we testing IO or running multiple processes on the same chip. If we do either of these, the Opteron's relative performance improves.

Many people commented that my previous benchmarks, which compared OS X on the G5 with Linux on Opteron chips, were limited because gcc is optimized for the x86 family. In these benchmarks, this excuse can obvious not be used. There are some serious issues with OS X and the gang in Cupertino should get to work. Even Windows XP performs better than OS X, which is embarrassing for Apple and a testament to Microsoft engineering given the legacy software requirements of Windows.

As noted before, the hybrid XNU kernel is probably to blame for OS X's problems. People on the web have recently been speculating whether Apple with drop the Mach micro-kernel portion of XNU. These rumors have picked with the departure of Avie Tevanian, an important figure in the development of the Mach Kernel first at Carnegie Mellon and then at Apple. Interestingly, Chris Emura, the Filesystem Development Manager within Apple's CoreOS organization, recently stated that Apple is interested in porting Sun's ZFS filesystem to OS X. If true, it may be that Apple is interested in fixing core issues with their operating system now that the eye candy is stable.

I have conducted many more benchmarks on these and other machines. For example, I have tested the HFS+ filesystem. It is slower than reiser especially for small and medium sized files and slower than XFS especially for large files. If you want these additional benchmarks, let me know.

There are claims on the web that when Apple developers compile OS X on the 970, they use -Os. That is, they optimize for size and not for performance. "So even though Apple talked a lot of smack about having a first-class 64-bit RISC workstation chip under the hood of their towers, in the end they were more concerned about OS X's bulging memory requirements than they were about The Snappy(TM)." AnandTech has an article which offers another explanation for why OS X is so inefficient. See No more mysteries: Apple's G5 versus x86, Mac OS X versus Linux.

A writeup of my previous benchmarks, which includes a review of my general impressions of OS X, is available HERE.

Suggestions

If you have any suggestions on how to fix the terrible performance of (this software on) OS X or if you think something here is erroneous, please contact me.

Links

See similar benchmarks available on AnandTech's website: "No more mysteries: Apple's G5 versus x86, Mac OS X versus Linux" and "No more mysteries, part two".

For another review see When a Linux user buys Apple's Mac mini.

[Echo Talon]

[Echo Talon]

Don't sweat it. This is just another perfect example of the Linux pushers claiming their foreign clone of an O/S is the greatest technological feat of all human history.

LOL! What exactly is OsX? At least with Linux you have a choice of hardware.

[Golden Eagle]

NeXT was looking that good over 10 years ago.

[Golden Eagle]

You can get Apple on PPC or x86. Solaris runs on SPARC or x86 too.

[Echo Talon]

NeXT was looking that good over 10 years ago.

What good? I have all the eye candy turned off. LoL. no need for it.

[Echo Talon]

Get OSX to run on my

Athlon64 3500+
Epox Ep-9NDA3+ mobo
ATI Radeon X800XT All-in-Wonder video card...

[SunkenCiv]

More and more audio streams are beoming inaccessible to my Mac.

Must be those sites have switched to proprietary "standards".

[tortoise]

Apple is more likely to switch to Solaris IMO. Did you see the announcement about ZFS?

Apple will never switch to Solaris, which while a good server OS in some respects (e.g. stability) is also a pig in others. ZFS is a pretty spiffy filesystem, but it performs poorly out of the box. Solaris on the desktop is pretty sucky even with all the extra bells and whistles thrown in. Linux+XFS on Opteron makes a pretty fine server platform, being both fast and relatively stable, without the crappy bedside manner that Solaris can have.

If they dumped Mach (which honestly is a turd), they most likely would use one of the monolithic *BSD kernels. The FreeBSD kernel is wicked fast and pretty clean, but does not scale to really large servers the way Linux can. But then, Apple has never been interested in anything but the low-end server market, perhaps a recognition of the limitations of their OS.

[quidnunc]

HAL9000 wrote:

It's a problem with the KRLA web page, but here is a workaround - Open Windows Media Player

Select the "Open URL..." command under the file menu

Paste this URL into the box -

http://boss.streamos.com/wmedia-live/kkla/7723/16_kkla-16_krla_0105_050106.asx

then click the OK button. You should receive KRLA live. You can add it to Favorites for a future shortcut.

I am much obliged to you, your work-around worked like a charm.

Out of curiosity, how did you dig out the boss.streamos URL?

I searched for it in the source code but couldn't locate it.

[HAL9000]

Out of curiosity, how did you dig out the boss.streamos URL?

I viewed the HTML source for the stream player window, found the URL for the stream, then removed an unnecessary parameter at the end of the URL. After that, the URL worked in Media Player, and I'm glad it's working for you too.

[Echo Talon]

Case $65
Power Supply $90
Motherboard $123
AMD Athlon 64 X2 4800+ 2.4ghz Dual Core 2x1MB retail box $632
2 x 300GB Seagate Hard drives raid 0 $200
Video card $400
Memory 2GB $190
2 x 16x dual layer dvd burners $70
1 x floppy $12
Total: 1,782

[ThinkDifferent]

And your time is free of course...but yeah, G5 towers are currently overpriced. This may be because anybody who can wait for the Intel towers is doing so, and the professionals who need G5s today will pay. iMacs and minis are much more competitive.

2 x 300GB Seagate Hard drives raid 0

No thanks, I occasionally like to read my data after writing it :)

[Echo Talon]

2 x 300GB Seagate Hard drives raid 0

No thanks, I occasionally like to read my data after writing it :)

huh? you have a problem with RAID? ok then just 2 x 300GB hard drives without raid... your choice. Hell get 4 of them for $400 still be cheaper than than mac and Put it in RAID 0+1 Stripping+Mirroring(for security)

[Echo Talon]

or get one of these and still be cheaper than the Mac here

[tortoise]

I'm curious: how much porn do you have to download to fill 600GB of disk space? ;-)

For my purposes, I buy the smallest practical drives I find and mirror them. The high water mark for my various systems is currently something like 40GB, with music being the biggest consumer of that space. Big disks are dreadfully slow for anything I'd use big volumes for anyway (like reconstructing a database snapshot). It is good to know someone is buying all these big hard drives. Me, I am whore for RAM, the more the better. I do not think I've had a system with less than a gigabyte in 6-7 years.

[Echo Talon]

Maybe you should read some hardware reviews, actually the larger drives are faster than the smaller ones... Do some video editing and you will see the need for space.

[Echo Talon]

Yep, that Mac only comes with 512mb ram... sheesh... the system I just slapped togeather has 2GB(2048mb)

[Echo Talon]

Oh, also you do know that their is a thing called disk partitions right? :)

[Californiajones]

There's a way to get Windows Media on your Mac. Google it or go to a mac site.

[Echo Talon]

OK, if you don't want a bunch of space but want blazing speed just get this... here