Posted on 08/15/2007 9:01:23 AM PDT by N3WBI3
Bloomberg believe VMware’s IPO today may the largest technology offering since Google. But doubts have been cast over the company’s supposedly proprietary ESX product, which may be derived from Linux.
Updated:
Added link to archived copies of Torvalds’ quote regarding hooks to binary modules.
Clarified that ESX has two kernels that run directly on the hardware (some readers [...]';
Bloomberg believe VMware’s IPO today may the largest technology offering since Google. But doubts have been cast over the company’s supposedly proprietary ESX product, which may be derived from Linux.
Updated:
For those short on time, this article is not about VMware providing source for the Linux kernel used by ESX - this can already be downloaded from VMware.
It is about whether vmkernel, a proprietary blob that can only be loaded by a Linux kernel, can be considered a derived work of Linux.
VMware recreated the modern virtualization industry, resurging an idea last heard of in the 80s to allow multiple instances of an Operating System run simultaneously on the same hardware, on today’s commodity systems.
VMware has 55% market share and few competitors: Microsoft’s competing Veridian virtualization originally scheduled for this years Windows Server release has slipped to 2009, and Xen, supported by IBM, Red Hat and Sun, has yet to get into the hands of their customers en masse.
VMware owns this market right now, and even if VMware’s share shrinks - and nobody’s saying that - Garner estimates VMs will grow from 5 to 40% of new Operating System installs will be on VMs by 2009. Their IPO this week promises to be one of the biggest since the early 2000s.
VMware’s main product is ESX Server. Unlike desktop Virtualization products, ESX is installed directly on to the hardware. ESX is an OS designed only to run VMs and tuned to run them as fast as possible. The VMs live on storage shared between ESX servers, and it’s possible to move running VMs betweens ESX servers. Need some more processing power? Plug a new server in, install ESX, and drag some VMs there to start using it – without ever stopping the apps.
It’s simple to use, avoids all kinds of downtime, and consolidates hardware assets to better utilise them. It’s not cheap, though, and neither are the SANs made by VMware’s parent company EMC.
VMware leads a rapidly growing market with a proprietary product. That’s the basis of estimated value of at least 10 billion dollars. But is VMware ESX really proprietary?
Looking at an ESX server, you’ll find what looks like a Linux OS. This isn’t a secret – VMware call this the ‘console OS’. Is ESX server based on Linux? The VMware ESX FAQ provides an answer:
‘Q. Does ESX Server Run on Linux? On Windows?
A. ESX Server runs natively on server hardware, without a host operating system. ‘
Ok, so ESX doesn’t need a host OS. It’s pretty clear that ESX installs directly on the hardware without needing Windows, Linux or any other OS installed first – ESX itself is the OS. The question then is whether the ESX OS is based on Linux. The FAQ continues:
‘The ESX Server virtualization layer is a highly compact and efficient operating system kernel entirely developed by VMware for optimum virtual machine performance … ‘
Or, as the Wikipedia discussion for this issue states:
‘ESX’s kernel is definitely NOT Linux’
Let’s hear it from the horse’s mouth. VMware employee Jim Troyer further states in the discussion for this article:
‘ESX Server’s vmkernel is definitely not Linux or derived from Linux. ‘
A kernel is the core of an Operating System. Kernels do basic things like give memory and CPU time to apps. Windows has a kernel called NTOSKRNL, Linux distributions use the Linux kernel. And apparently VMware ESX uses VMWare’s own kernel, called VMkernel.
So ESX does not use Linux. Matter settled.
Or does it?
Back to that FAQ:
‘ESX Server also incorporates a service console based on a Linux 2.4 kernel that is used to boot the ESX Server virtualization layer. It also runs ESX Server administration applications.’
So according to VMware, ESX actually has two kernels that run directly on hardware - the vmkernel, and a Linux kernel. This sounds a bit odd, given a computer can only run one kernel at a given time – otherwise which one determines who gets access to the CPU, memory, and other hardware?
But VMware explain there is a Linux kernel, but it’s only used to boot the other kernel. In this interview Michael DiPetrillo, VMware’s Senior Systems Engineer, states at 4:44 ‘In the 2.xs (the Service Console) was home to a reasonably large number of things, bootloader, it had it’s own networking‘.
John Troyer, Senior Product Manager of the VMware Technology Network, mentions on Wikipedia’s discussion page:
‘A good citation for the relationship between the vmkernel and the Service Console is Chapter 2 of Oglesby, Herrod, and Laverick’s ESX Server Advanced Technical Design Guide‘
Which also states that Linux ‘acts as the bootstrap for the VMkernel’ (bootstrap is another term for bootloader.
A bootloader is a small piece of software used to start a kernel. Every OS had one:
But wait a moment: this might be usual, but it’s not unique - Novell Netware used the similarly large MS DOS kernel to load itself. And the document goes on to state that ‘Once the COS has loaded ESX, the VMkernel will assuming the role of primary operating system… then load the COS and several other “helper worlds” as privileged VMs.’
Hrm, it seems that Linux is used to load vmkernel, and then vmkernel virtualizes Linux.
Let’s see how that’s done.
vmkernel is loaded by vmkmod, which itself is loaded from the ‘vmware’ service at boot time. Linux services are started via scripts. The specific line in the vmware service script that loads vmkernel runs:
insmod vmkmod
‘insmod’ installs (loads) Linux driver modules into memory. It doesn’t load anything else but driver modules.
When ESX boots, Linux is ESXs kernel: vmkmod is a driver, and vmkernel a large piece of software loaded by that driver that functions in kernelspace.
The license for the Linux kernel is quite different to the licensing for DOS that allowed Netware to use it for a bootloader. Proprietary drivers for Linux kernel have an interesting licensing situation. Unlike the license for higher level libraries, which allow those libraries to be used by both Open Source and proprietary software, the license for the Linux kernel specifies that software based on the Linux kernel must be licensed under the same license.
As a result of that, most drivers for Linux are Open Source: whether created by coders working for Linux companies like Red Hat or IBM, hardware companies like Intel, or enthusiasts who want to make sure their hardware works well under the OS, most Linux drivers are Open Source.
But some proprietary modules do exist, and they do that on one premise: Linus Torvalds (the copyright holder for the Linux kernel) has repeatedly stated that he doesn’t consider drivers ported from other operating systems to be derived works of Linux. After all, if something can load without Linux, it can’t really be considered a derived work. NVIDIAs Linux video card driver shares most of its code with it’s Windows equivalent – NVIDIA’s Linux driver is proprietary, and according to Torvalds, that’s fine - since the driver clearly doesn’t need Linux to start it.
Witness Alexandro Rubini’s interview with Linus Torvalds, published in the September 1998 issue of Linux Gazette.
Alessandro: ‘What is your position about the availability of Linux modules in binary-only form?’
Linus: ‘I kind of accept them, but I never support them and I don’t like them. The reason I accept binary-only modules at all is that, in many cases, you have, for example, a device driver that is not written for Linux at all, but, for example, works on … other operating systems, and the manufacturer … wants to port that driver to Linux. But because that driver was obviously not derived from Linux (it had a life of its own regardless of any Linux development), I didn’t feel that I had the moral right to require that it be put under the GPL, so the binary-only module interface allows those kinds of modules to exist and work with Linux.
That doesn’t mean that I would accept just any kind of binary-only module: there are cases where something would be so obviously Linux-specific that it simply wouldn’t make sense without the Linux kernel. In those cases, it would also obviously be a derived work, and as such the above excuses don’t really apply any more, and it falls under the GPL license.’
VMWare’s desktop products don’t use the driver, as they don’t need the advanced storage functions that the vmkernel does. The only way to load vmkernel is by vmkmod, a driver that requires Linux. Take away Linux and there’s no way to load vmkmod and start vmkernel.
It’s possible to ditch, remove, or crash Linux after vmkernel has virtualized it - but you wouldn’t be able to get to that stage without Linux being used to load vmkernel.
But wait - perhaps vmkmod might be a derived work, but the vmkernel it loads is stored in a separate file. Lets check out Torvalds again, from 19 Oct 2001:
‘I personally consider anything a “derived work” that needs special hooks in the kernel to function with Linux (i.e., it is not acceptable to make a small piece of GPL-code as a hook for the larger piece), as that obviously implies that the bigger module needs “help” from the main kernel.’
ESX Server 2.5 architecture, from the VMware Advanced Technical Design Guide. VMkernel uses the vmkmod hook to work with Linux (Console OS). According to Linus Torvalds, such applications are derived works of Linux.
So why hasn’t this issue come up before?
Christopher Helwig is the Linux SCSI storage maintainer, and one of the top 10 contributors to the Linux kernel. He has been pursuing VMware over the issue for a year. Why hasn’t Hellwig pursued VMware legally? Hellwig answers in a post to iSCSI mailing list:
‘VMware uses a badly hacked 2.4 kernel with a big binary blob hooked into it, giving a derived work of the Linux kernel that’s not legally redistributable. I unfortunately don’t have enough copyrights on that particular version to sue them. I do object to use of any open-iscsi code of my origin to be used with it, though.’
Do VMware know about this? They should: they employ a number of developers who work on the Linux kernel. And they do: witness the following from last August - replying to a post from VMware’s Zachary Amsden to the Linux kernel mailing list, Hellwig wrote to Zachary:
‘Until you stop violating our copyrights with the VMWare ESX support nothing is going to be supported. So could you please stop abusing the Linux code illegally in your project so I don’t have to sue you, or at least piss off and don’t expect us to support you in violating our copyrights.
I know this isn’t your fault, but please get the VMware/EMC legal department to fix it up first.’
Zach did not post a reply, and no mention of the topic has come from VMware anywhere, despite Hellwig’s continued claim on the list in front of VMware staff. From May 6 this year: ‘VMware… are on the almost black side of the illegality’
Is Hellwig right, and is VMware ESX a derived product of Linux? Unless vmkernel can be loaded without the Linux kernel, it would appear so. VMware was developed from another, long ago OS created as a research project, but it’s unclear whether vmkernel was ported from that OS or rewritten as the Linux-requiring binary blob.
What’s more of an issue is that VMware had these serious questions posed directly to them a year ago, repeated in a public forum many times since, but have yet to respond at all.
ESX is VMware’s main product. EMC are hoping to raise 10 billion dollars from its promise. They have a duty to defend it, and clarify the licensing situation for all concerned.
OSS PING
This was covered on Slashdot.org quite a bit yesterday. The answer that satisfied me was that Linux kernel can actually stop itself after loading a completely foreign kernel, then pass control over to this second kernel which runs everything.
The Service Console is a version of Linux that runs on top of vmkernel and lets you configure and control other VMs.
I know Cisco infringed on the copyrights to various free software, but I've never heard of open source infringement on the part of the others. Microsoft still has BSD code in Windows, but it complies with the BSD license. Apple has available all of the open source code they're required to give (like GPL) and even some they're not (like BSD). Don't know much about Oracle and open source though.
Can you fill me in?
That's general ideas, not software. Besides, Apple paid for the trip and released a product that was much improved over Xerox's basic UI ideas.
Nevermind, re-reading the post I see “ideas” was included.
With respect to Oracle, I commend to your attention the July 24, 2007, edition of Wired Magazine, wherein is to be found a long, interesting article entitled "Inside the High Tech Hunt for a Missing Silicon Valley Legend". It's about the Microsoft database wizard, Jim Gray, who disappeared on what was supposed to be a brief boat trip on January 28, 2007. Here are the money grafs:
Gray enrolled at UC Berkeley in 1961, grew his hair long, and rented a basement room with a dirt floor for $5 a month. He thought about majoring in philosophy but lost interest when he discovered computers. He signed up for graduate classes without taking the prerequisites, and in 1969, he earned the university's first PhD in computer science.After a stint at Bell Labs, the scruffy young coder took a job with IBM Research in San Jose. He gamely plunged into the nuts and bolts of data management, considered a dreary backwater even by avid readers of the Systems Journal. Back then, inputting a request to a data bank (the word database hadn't caught on yet) required a thicket of query syntax and a team of programmers. But in 1969, an IBM mathematician named Ted Codd had an epiphany. He sketched out a model for a so-called relational data bank that would make it easier to sift relevant needles from haystacks of information.
Gray joined an R&D team that was attempting to turn these theories into functioning software. His work focused on transactions: When should changes be firmly committed to the database? What if one of a pair of transactions fails halfway through? How can two (or 2,000) users access the same data simultaneously without corrupting it? In his research papers, he compared transactions to a marriage contract: What was the right moment to say "I do"?
Gray's solutions to these riddles are now embedded throughout the networked world, from Amazon.com to the corner ATM. The data bank he created with his colleagues, called System R, laid the technical foundations for a new industry. Updates of its query language, SQL, are still widely used. Compared to their clunky predecessors, relational databases turned out to be better suited to the revolution just over the horizon — GUIs, parallel processing, and the Web.
System R also inspired a seminal open source project at UC Berkeley, a relational database called Ingres, built by a crew led by Mike Stonebraker, now a professor at MIT. Though the teams at IBM and Berkeley were officially competing for the same small market, Stonebraker recalls, "Jim was unbelievably smart and very willing to help without getting credit. Most people considered business data-processing to be beneath them. Who would help Bank of America cash checks? No one foresaw how essential databases would become."
No one but an aspiring entrepreneur named Larry Ellison, then a young programmer working at Ampex on a data-storage scheme for the CIA. The agency's codename for the project: Oracle.
In 1977, Ellison launched a startup called Software Development Laboratories, using System R's papers as the blueprint for his own software. Then he repositioned his database for the emerging minicomputer market, betting that Big Blue would drag its heels. Ellison bet right, and the rest is Silicon Valley history.
"There was no love lost between Jim Gray and Larry Ellison. They hated each other," one of Gray's Microsoft colleagues says. But Gray was gracious in public, and he mellowed with the years. He told an interviewer in 2005 that his life had been a "researcher's dream — you have a lot of fun, you do something innovative, and then people make billions of dollars off of it."
No surprise those green party leftists at the Free Software Foundation are trying to confiscate code from others again. They set the trap by claiming their software is “free” but if anyone uses it to make any money expect them to come crashing your door down that you violated their license, as in this case even when they’re wrong.
They set the trap by claiming their software is “free” but if anyone uses it to make any money expect them to come crashing your door down that you violated their license
Can you post a link to FSF going after RedHat or Google who make money on GPL software? FSF goes after people who (a) Cant Read, (b) Dont Read, or (c) Try to steal copyrighted material.
No it’s a perfect example they’ll make those accusations even without proof. VMware had a successful IPO so the software communists come out to smear them and demand they hand their software over.
HA! The man who spent countless hors here defending SCO can say this! My goodness the hubris of this post is without equal!
VMware had a successful IPO so the software communists come out to smear them and demand they hand their software over.
Refusing to defend the copyright holders rights I see, typical..
They set the trap by claiming their software is “free” but **if anyone** uses it to make any money expect them to come crashing your door down that you violated their license
-
Can you post a link to FSF going after RedHat or Google who make money on GPL software? FSF goes after people who (a) Cant Read, (b) Dont Read, or (c) Try to steal copyrighted material.
Eben Moglen has been very critical of Google for not contributing more especially lately, and even hinted future versions of the GPL may require you to give up your code even if you don’t distribute it to end users. Red Hat has also been criticized for not handing all of their tools over either, I don’t know what planet you’ve been on but the FSF doesn’t approve of ANY proprietary or closed software, their exact mission is to make all software their definition of “free”. Most companies that make money don’t feel that way of course, but that’s why you have these constant attacks from the FSF. Did they take the streets in their hazmat suits yet LOL?
Is that banging down the door claiming a license violation?
hinted future versions of the GPL may require you to give up your code even if you don’t distribute it to end users.
At which point Ill join you on the anti GPL.X side of the fence
. Red Hat has also been criticized for not handing all of their tools over either
Is that banging down the door claiming a license violation?
I don’t know what planet you’ve been on but the FSF doesn’t approve of ANY proprietary or closed software
Never said they did, I said their license does not trap the literate. You said they bang down the door of any gpl software user who does not give back, what planet are you living on?
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.