Desperately Seeking Xen
June 27, 2007 
Feature Article By Jeff Gould
Copyright © 2007, Peerstone Research Inc, All rights reserved.
June 27, 2007
What's going on with Xen, the open source hypervisor that was supposed to give VMware a run for its money? I can't remember how many IT trade press articles, blog posts and vendor white papers I've read about Xen in the last few years. If I had a dollar for every piece ever published about Xen, I'd be... well, not quite as rich as the Google kids, but still very well off. The vast majority of those articles - including a few I've written myself - take it as an article of faith that Xen's paravirtualizing technical approach and open source business model are inherently superior to the closed source alternatives from VMware or Microsoft.
Suffice it to say there has been a lot of excitement and optimism in the pundit community about Xen. As recently as last January, influential blogger-journalist Daivd Berlind flat out predicted that:
"XenSource will eventually overtake VMWare in terms of market share..."
But now that Novell and Red Hat have both been shipping Xen in their commercial Linux distributions for some months, things have grown eerily quiet. Sure, there is still product news coming out of the Xen vendors, and we'll get to that in a moment. But what I'd really like to know is - who's actually using this stuff in production? And I mean actual end-user organizations, not ISPs or hosters. Based on the absence of Xen-related chatter, my guess is that production users of Xen are still few and far between.
The first thing a new technology like Xen needs before it can win over mainstream corporate buyers is a credible, reasonably long list of reference customers. You will look in vain for such a thing on the Red Hat or Novell web sites. Novell has put out exactly two press releases this year about Suse Linux customers who are using Xen, both very skimpy on details. Red Hat published something last year about how thrilled its RHEL4 customer Priceline was about the promised future capabilities of Xen. But that was before they actually got around to shipping Xen with RHEL5 in March. Since then, radio silence.
To be sure, the small VC-funded companies that are selling proprietary management tools for Xen - for example, XenSource and Virtual Iron - do mention a handful of customers on their web sites. But no more than a handful. Taking Novell, Red Hat, XenSource and Virtual Iron together, you'd be hard pressed to come up with a dozen named reference customers for Xen. Pretty thin gruel for a piece of software that has caused so much ink to be spilled, or should I say, page views to be generated.
I don't doubt that hundreds or perhaps even thousands of Linux geeks around the world are kicking the tires of Xen, especially those using the free distributions that include it, like Fedora. But that doesn't make Xen a serious threat to its rivals in actual deployment in real-world data centers, at least not yet. By any measure, VMware is still the 8,000 pound gorilla in this market.
For the sake of comparison, let's remind ourselves just how big VMware's footprint in the market really is. In the first quarter of 2007, the firm - soon to be partially spun off from the EMC mothership - reported license revenue of $170 million, up 84% year-on-year. Assume an average selling price of a little over $3,000 per copy for the flagship ESX hypervisor. That means VMware probably shipped more than 50,000 hypervisors in the first quarter alone, not counting all the free downloads of VMware Server (which, like Microsoft's equally free Virtual Server, is not a true "bare metal" hypervisor). VMware says it has more than 20,000 customers in the world. Based on a back-of-the-envelope extrapolation from the historical revenue numbers published in the company's April SEC S1 filing, there should be somewhere between 250,000 and 300,000 ESX licenses out there. This corresponds to roughly 1% of the world server installed base, an impressively big number. And more copies are being added every day. Looking at VMware's Q1 results, it's a safe bet that its installed base will at least double this year.
For those who believe that Xen will one day overtake VMware, these numbers set the bar pretty high. For the moment, there is nothing to suggest that Xen is achieving anywhere near the growth rate it would need to perform such a feat, even in the distant future. Of course, all great things start out small, and there is still plenty of time for Xen, since we are by all indications only at the beginning of the great virtualization wave.
But still, there is no reason to take Xen's triumph as an inevitable article of faith. If we take off our open source blinders for a moment and look at Xen objectively, we can begin to spot a few of the flaws that appear to be holding it back.
The first problem with Xen is that as a piece of software it is far less mature than VMware ESX. eWeek began its recent lab evaluation of Red Hat Enterprise Linux 5 (RHEL5, the first Red Hat version to incorporate Xen) with the sober warning that:
"... companies in search of an out-of-the-box server virtualization solution should not expect it here..."
The eWeek reviewer goes on to compare the rudimentary open source management tools Red Hat provides for Xen with the fancier proprietary stuff from the VC-funded Xen startups or VMware:
"Compared with VMware's VI3 (VMware Infrastructure 3) and with the Xen-based Virtual Iron and XenEnterprise products we've reviewed, RHEL 5's tools for creating and managing guest machines are pretty Spartan, and our experiences installing and running Windows Server 2003 and RHEL 5 guests contained more troubleshooting and Googling than we would have liked."
Red Hat is continuing to work on its GUI-based Xen installation tool virt-manager, and the just released Fedora 7 has a preview of what the future RHEL 5.1 will look like. But it seems that some assembly is still required to get Xen virtual machines up and running. For example, Fedora reviewers have run into a thicket of configuration issues:
"The Fedora team has followed the XenSource model and begun to store all VM configuration details in a database, referred to as xenstore. This enables the rest of the tools to use one consistent resource for reading configuration details, but also serves to complicate configuration changes. For example, virt-manager now shows all configured virtual machines, whether they are running or not (it used to only show running ones), which enables starting of paused or off virtual machines from within the tool. The bummer is that a complex configuration requires the manipulation of the database through a series of command line tools. Fedora team has eased this a bit through virsh (a command line tool that leverages libvirt)... to get an editable xml representation of the vm configuration... you can still use the standard Xen "xm" commands with a config file to create/start a vm, but these won't be added to the database, so when they stop, they disappear from virt-manager."
Oh my. Editable XML configuration files, obscure command line interfaces, grayed out options in the GUI? Thanks, but no thanks. This thing doesn't sound like it's ready for prime time in Data Center USA.
Of course Red Hat is aware of these maturity issues, and in recent months it has been notably muted in its public hyping of Xen, in marked contrast to last year's big PR run up to the release of RHEL5. It may even be that the Linux distributor's commitment to Xen is not as deep as advertised. To avoid lock-in, Red Hat's engineers have cleverly adopted the strategy of isolating Xen behind an abstraction layer called libvirt. This API can be used to invoke hypervisors other then Xen, such as the open source x86 instruction set emulator Qemu, or the Kernel-based Virtual Machine (KVM), a new open source hypervisor sponsored by a startup called Qumranet. Interestingly enough, Qumranet's co-founder is none other than former XenSource CTO Moshe Bar, and KVM has actually managed to get itself incorporated into the official Linux kernel (with version 2.6.20) ahead of Xen, despite the latter's multi-year head start. So it seems that even on the open source front Xen will have some real competition.
Then there is the question of performance. Xen's promoters have spent the past four years hammering into our heads the notion that paravirtualization is inherently faster than the full virtualization used by VMware. All hypervisors designed to run on x86 chips must cope with the fact that this architecture was simply not designed to make virtualization easy. This is in marked contrast to the competing server chips from the proprietary Unix vendors, like IBM's Power or Sun's UltraSparc. The latter chips carefully segregate instructions available to any random user program from those restricted to the operating system kernel itself. The x86 chips on the other hand blur this distinction, with the result that certain critical instructions produce one result when invoked by the OS kernel and a different result when invoked by a user application. This makes the traditional trap-and-emulate virtualization approach developed by IBM more than thirty years ago impossible. An x86 virtualization scheme must somehow intercept these errant instructions before they wreak havoc, and hypervisor designers have a choice of two ways to do this. Either their hypervisor must translate the offending binary code on the fly while the guest OS is running, or else they must modify the ambiguous instructions in the original source code of the guest OS prior to compilation. VMware pioneered the first approach, while Xen and Microsoft's future Viridian hypervisor adopt the second.
On the face of it, binary translation sure sounds like it would make for slower hypervisor performance than paravirtualization. After all, the latter approach lets time-critical functions like device drivers run as if they were native code. But despite this advantage on paper, the few published benchmarks comparing Xen and VMware are ambiguous. It must be noted here that VMware employs the scandalous policy of wording its user license so as to forbid the publication of benchmark data it doesn't approve of. This arrogant behavior can only be characterized as a self-awarded license to deceive the customer, because it limits competitors like XenSource to using only the performance data that VMware sees fit to allow into the public domain. Nevertheless, XenSource's own white paper on the subject suggests that Xen's paravirtualization approach produces only modest gains over ESX. In benchmark after benchmark the performance differences between Xen, ESX and native applications are usually only a few percentage points. The conclusion a reasonable observer might draw from this is that in most real-world cases these hypervisors are not really CPU-bound at all, but limited chiefly by available memory and disk input/output performance.
Another question hanging over Xen performance concerns the availability of paravirtualized drivers for Windows. As a practical matter, it seems that Xen in RHEL5 is mostly being used as a partitioning solution for hosting multiple virtual copies of RHEL5 or perhaps RHEL4 on a single RHEL5 physical machine. But the vast majority of data centers these days run Microsoft and Linux stacks side-by-side. Even Steve Ballmer has recognized this. One of VMware's strengths is that it works extremely well with both stacks, and observation suggests that a large number of VMware customers take full advantage of this fact.
Since Microsoft's OS kernel source code is not freely available for modification by third parties the way Linux is, Xen can't make the same performance promises for Windows Server that it can for RHEL or SLES. However, a developer with access to Microsoft's proprietary HyperCall APIs can in principle circumvent this problem by leveraging the virtualization-friendly architecture tweaks in recent Intel and AMD chips (known as Intel VT and AMD-V respectively). XenSource has already released its paravirtualized drivers for Windows Server, and Novell will have them when it ships SLES 10 SP1 in July. According to XenSource user Jim Klein (who doesn't cite his sources), Red Hat won't have these drivers until RHEL 5.2 is released sometime next year. Other bloggers have speculated that the proprietary nature of Microsoft's APIs will make it impossible for Red Hat to use them unless it signs some kind of deal with Redmond. However that may be, there is still no solid benchmark data to demonstrate that these drivers can actually move the Xen performance needle in a big way compared to VMware. But certainly Xen fans can always hope.
To be fair, the Xen story is really only just getting started. Given the incredible financial results that VMware has been racking up (on the eve of its IPO, VMware's revenue run rate is well over one billion dollars per year), there can be little doubt that Red Hat, Novell and XenSource have every incentive to make Xen competitive with ESX. But it remains an open question whether they will be able to do so soon enough to matter. It took VMware five years to reach 1,500 paying customers. Xen doesn't have that much time, though, because soon it will have to square off against a new competitor, Microsoft's Viridian hypervisor, which is scheduled to be incorporated into Windows Server 2008 within a few months of the new OS's planned release at the end of this year.
Don't confuse Viridian - aka Windows Server virtualization - with Microsoft's current Windows-hosted (non-bare-metal) offering called Virtual Server. Viridian is a brand new and very ambitious project that has a long list of features (albeit recently pared back) and is explicitly based on the same paravirtualization approach as Xen. Considering that Viridian will be coming to market an amazing full ten years after the first VMware bare metal hypervisor, it's not surprising that Redmond chose the more modern approach. There is already a lot of technical information about Viridian floating around on the web - Microsoft has been using its Windows Server Division Weblog to propagate the latest news. But since only a handful of early beta testers have actually seen it (the public beta has been delayed until later this year), it's pointless to speculate on how Viridian's performance or ease of management will shape up compared to Xen and VMware. If history is any guide, Microsoft will come out of the chute with a solid but not great product, and then deploy its unrivalled block-and-tackle ground game to gradually narrow the gap with the market leader. The examples of Windows Server 2003, SQL Server 2005 and Exchange Server 2007 are testimony to just how far they can get with this strategy.
Between Microsoft, Novell, Red Hat, XenSource, VMware and the scarcity of good benchmark data, it's going to be tricky for users to figure out which operating systems will deliver the best virtualization performance on whose version of which hypervisor. Microsoft has hired XenSource to develop a "shim" (API adapter) that will allow paravirtualized Linux to run on Viridian, and it has licensed its HyperCall API to Novell to allow that firm to develop a similar shim for Windows Server 2008 running on Xen. It isn't clear at this point how Red Hat will work with these shims, or even what its options will be, though it seems unlikely that geeky API issues like these will seriously slow Red Hat down in its march to build a commercial open source franchise covering the full stack from the metal on up.
The confusion about Xen performance and APIs may not matter all that much, because it is increasingly clear that Microsoft's true strategic rival in the virtualization arena is VMware, not Xen. Even if Viridian doesn't blow VMware out of the water - and that's a safe bet - it is very likely to squeeze Xen, and in doing so it will squeeze Linux. A year ago I thought that Xen offered Red Hat a huge opportunity to take some market share from Microsoft before the release of Longhorn (Windows Server 2008). Apparently Red Hat thought so too, because they planned and promoted the release of the all-important RHEL5 around Xen, even delaying the launch date by several months to smooth out the snags encountered in the integration effort.
But today it's apparent that however Xen evolves in the future, it isn't going to be the Longhorn killer Red Hat thought it would be.
Reader Comments (33)
Xen is fine technology and VMWare is great (I've been using it for quite a long time), but I am betting on KVM. Right now I bought a new version of VMWare since I found Xen less than ready for prime time.
I'm giving the open source community a few years to make KVM the technology to use. XEN, well I think it will always be the daughter that was the bride's maid and never the bride.
Great article, very clear and well written.
Also, I absolutely agree with what Chuck wrote above: KVM does indeed have the brighest future ahead.
For completeness, it's worth noting that Amazon's Elastic Compute Cloud (EC2) is entirely Xen-based. Depending on who you talk to there's upwards of 1,000 host nodes in EC2, each running 4-8 guests.
I am using OpenVZ everywhere. If you are using the same OS, then you can't beat it for performance. It is also quite mature.
As with many other things in the world of Linux and OSS, the hype over virtualization solutions doesn't reflect the market realities.
Free/OSS is often a non-starter in many enterprise settings,because where the limiting factor is performance not price, companies want the best technology (with a mature company behind it), because its usability, features, stability, and support etc. are ultimately crucial to their business -- thus they are willing to pay for
the best overall solution.
Companies that have the right management can leverage the profits of a commercial solution to: hire the best people, support huge Q&A, testing and vendor certification programs, do usability testing and develop a high quality UI, employ product managers who can constantly keep in touch with the needs of customers and guide the product, and fund acquasition of new startups as well as supporting in house research and development to keep the company innovating.
VMware has a singularity of purpose and had ten years to build up a mature company with a long track record of inovating and shipping product, and a huge war chest.
In the long run, when it comes to virtualizing commodity hardware in the enterprise, Microsoft is really the only potential competitor. No one else has the resources and the focus to take on a competitor that is already so well established both in terms of product maturity and funding.
Even at that, it will be years before we get to see if there is even a ball game here. In other areas, such as RDBMS, microsoft has been in the game for well over a decade (getting closer to 2), and still only commands perhaps 15% of the market against more focused and established players like Oracle and IBM.
Xen has been in production for over 18 months at Oracle supporting revenue generating business. They're running over a dozen different OSes including Windows on VT and Pacifica HW.
KVM is certainly going to part of the future, but anyone that's done extensive work with all of the VM options out there knows (or should) that paravirtualization is in most cases significantly faster than full virtualization when running mixed OSes. Xen has long legs and will continue to be used in production environments for years to come.
XenSource has 500+ customers using their commercial Xen based platforms. A good percentage are using this stuff in production.
Check out www.slicehost.com - they're running Xen-based VMs for developer hosting. Pretty cool, and pretty darn stable and responsive too.
Ultimately, from an end user point of view and from an admin's point of view:
1) Stability and Reliability without needing to become a guru.
Out of the box, XEN fails miserably at this. I've been admining *nix for quite a few years and find the Xen approach to be painful for anyone not familiar with the Unix CLI and with the concepts put forth by the virtualization community. The GUI that RH has slapped onto the system is nice, but ultimately, doesn't help much when something goes wrong. And for me, getting Linux/Windows/anything to run on Xen fresh out of the box resulted in two possibilities: VM crash or Host OS crash.
The need to Google for an answer to what is essentially the "opening the box and giving it a test spin" is pretty bad and only takes away from the future of XEN. In stark contrast, the VMware product just works out of the box and is replete with help and example scenarios that help you to figure out what went wrong, instead of needing to resort to Google, except for the most vexing of problems... and even then, you will find that the solution is on VMware's forums.
2) IT Manageability
Where the out-of-box tools to create and start a VM for XEN are primitive, the tools to manage large collections of Xen servers and their respective virtualized guests is even weaker. To the point of being non-existent. The setups at Amazon EC2 and at all of the web hosting companies and ISP(s)... how many of those are heavily modified an in-house tools? And how many of them are out-of-the-box tools you get with Xen?
That gap in the Xen toolset is telling... we aren't talking about a race with two cars on the strip ready to take one another on. We're talking about a tricked out car on one side, with a complete team of technicians working on the car to ensure it runs right... and on the other side(Xen), we are talking about a large group of people standing around a chassis and an engine, wondering how best to hook things up and get it running. The parts are there, there's just no good or agreed upon way to putting things together. No unifying mechanism for getting things to run.
3) The need for modified OS(s)/hooks
The weakness of the XEN method is that it requires the guest to be modified. This greatly reduces the adoption rate and means that for enterprise environments, there will be significantly less buy-in. People HATE to change what works. And if you have XYZ app running on ABC OS and platform, you aren't going to want to chance running on ABC-modified-untested, especially if that app is revenue generating.
The cases where XEN is heavily used is for web hosting applications, where the number of virtual machines are high, but they are all employing Linux or BSD, since those are easier to recompile and customize. Oracle has a vested interest in creating their own OS and probably see XEN extensions as a great way to leapfrog and make their database virtualizable. But who else, other than ISP/webhosting providers are using XEN in serious production environments?
When I talk to folks about virtuzliation, it has almost always been:
Vmware Server/Workstation for dev/test/production, because it's easier to manage and because we can run our OS's AS-IS in the VM. ESX/VI for their enterprise datacenters where hundreds, if not thousands, of VM(s) are running.
Xen for test systems, where someone has requested it specifically, or a commercial varient, when someone is setting up a multi-web hosting environment and wants something more powerful than just chroot, but doesn't want to spin up another box/server.
XEN needs to change that perception. XEN needs to get their toolset in place. XEN needs to make their software product easier and more reliable to use, out of the box.
Xen is getting left in the dust, as more and more people use VMware and MS's vrtualization products.
The silence is deafening.
Virtualization tweaks to devices, coming in the next year or so, should render the need for paravirtualized drivers obsolete. It will greatly improve overall performance, level the technological playing field between Xen and VMWare, and make supporting more operating systems much easier -- for those systems with the new hardware.
In the long run, when it comes to virtualizing commodity hardware in the enterprise, Microsoft is really the only potential competitor. No one else has the resources and the focus to take on a competitor that is already so well established both in terms of product maturity and funding
IBM?
you're onto something, I think.
I run a small ASP and started experimenting with virtualization this year. I don't think the primary benefit of virtualization will be in the corporate data center, I think you're going to see a thinning out of edge computing power physically located in a company's wiring room. I see the benefits of virtualization in managing large numbers of virtual servers and at that point you may as well locate the data in a data center with cheap bandwidth. I think that eventually we're going to see a situation where developers/sysadmins at a company have a test box on their dsl line then deploy the software to dedicated or colo equipment at remote isps. I think that this will accelerate, especially since virtualization reduces the cost for a dedicated-server-like experience.
Re the specific technologies, I think that in practice vmware is the best. I have a centos virtual image running in the background on my xp laptop. when I have to do a pitch for a suit the windows browser is surfing the vmware linux server in the background.
I bought a subscription for a vps running xen. it as a 384MB RAM 40GB HD xen fedora core 6 virtual machine. I found the virtual machine sluggish to wake up but then relatively ok while running. Subjectively and without controlling for various factors like the upstream ISP's sharing load factors etc, I think that vmware is faster than xen. It is certainly easier to deploy and more reliable as you do not need a custom version of the kernel etc.
I'm now trying another paravirtualization vps, this time under virtuozzo running centos 5. I'll let you know how that experiment turns out.
Bringing this all back to Xen, I think that it is crappy from a corporate perspective but better from a licensing/isp perspective. So I think we will see vmware dominate in the corporate wiring room and xen dominate in the isp space. and those two will converge when the corporate types move their stuff to isp servers running xen.
I suppose SAP, Oracle, Business Objects, etc gained traction in the corporate world because of their ease of use...
First and foremost, there is a VAST difference in functionality, user management interfaces, robustness, stability and performance between XenSource and all others leveraging the Xen technology, including the Xen Open Source itself. XenSource is not "just another vendor." They have the most code contribution to the open source and will continue to stablize this hypervisor technology where other Xen "vendors" like Red Hat, Novell and Virtual Iron, butcher its code and performance.
I have an extensive test/dev environment and am entertained by this emerging technology. I have recently run head to head comparisons of RH, Novell, Virtual Iron, MSFT VS, VMware's VI3 against XenSource's XenEnterprise. Though there is no downcasting VMware's maturity and product stability, it lacks a bit of freedom and the performance of XenEnterprise. The other players using Xen like RH and Novell, being operating systems themselves, will understandably show performance delays with all other OSs other than their own. However both run great and leverage this raw open source well. Virtual Iron was frankly a waste of my time. They should outsource XenSource for their development next time.
Overall, XenEnterprise is a stable and robust product. Though XenSource does lack VMWare-like maturity, it does boast of 500 unique customers in 6 months of product avaialability. It took VMWare to 5 years to make 1500. I wouldnt discount XenSource's ability to become a very real player.
Thats my 2 cents.
"Microsoft is really the only potential competitor. No one else has the resources and the focus to take on a competitor that is already so well established both in terms of product maturity and funding."
I've heard that one before (about 10 years ago), but look at where Linux is now.
Personally I use linux-vserver. The overhead is so low you can't measure it, it integrates with the rest of the system perfectly smoothly and it is Free (not only as in beer, but as in speach as well). If you want to run Linux guests you really can't do better (IMHO). I know there are equivalents for some of the other UNIXes, and given the degree of compatability / portability between them; this kind of thing pretty much solves the *NIX virtualisation question.
And then there is Windows. Anything except a full interpretation based virtualisation system is always going to be subject to the whims of Microsoft. They can (and likely will) kill any product they think is a threat by simply tweaking Windows so it no longer works, and the Windows world being what it is, it will be the virtualisation tech; not Microsoft who feel the impact of this.
OS level virtualisation or full virtualisatioin; anything inbetween is going to be outclassed by the *NIX OS level virtualisation system and at the mercy of Microsoft.
My scenario is this;
I was dying to use Xen in production @ my small 4 "server" organization. I ended up using VMWare under a stripped down "too bad I had to install X" Ubuntu 7 host to consolidate these servers. I would have much preferred Xen's bare-metal solution... but unfortunately my CPUs didn't have native virtualization instructions (older Xeon 3.0s without IntelVT) to support virtualizing the Microsoft Windows Server 2003 environment I'm stuck with. As I'm a use-what-you-have-that's-reasonable kind of guy, I didn't request buying new hardware... as this was a recent machine my predecessor purchased.
That said, I think this article is too quick to doubt Xen's growth. Virtualization will become commonplace over the next *few years* ... as hardware & the edge of the frontier move elsewhere, and I expect Xen to steadily gain market share in this. If anything, we at least owe it to them for igniting this market and scaring the pants off their "competitors".
XenGuy, what was your issues with VI (if you don't mind elaborating)?
Well as a user of XEN I've got to say I love the product. I looked at XenSource for my company and to be honest there were a few things I just didn't like about it when it came to the commercial tool but these were from an SA point of view as opposed to the technology and implementation. Like a lot of people I run some nice HP/Compaq systems with 2 disks mirrored for the OS and the rest in either a RAID 0+1 or 5 configuration which just works while providing me with maximum performance and disk space, particularly useful with 6 or 8 disks of course if you're using 360 or 380G5s etc.
But then along comes the XenSource product which says great, I've got tonnes of space, I'll use 4GB for my operating system and the rest I'll use LVM and just join the lot together. So now if I lose a single disk performance is potentially impacted across the whole environment, and if I lost another then I'm completely screwed depending on the disk lost. I've spoken to XenSource and one of their engineers recogised this fact but at the moment there is no easy way to solve the problem. Its a case of me manually rebuilding the system and using the good old DD tool or similar to manually adjust my system. Nice but not so great thank you!
But then along comes Centos for us as this is our preferred platform of choice. So I do my partitioning the way I like it with a huge data area for my virtual machines and its great. Life is good! And it just works beautifully!
And as for production. Well we're going live with this setup in the next 4 - 6 weeks having done a huge amount of testing / building of VMs across a variety of systems including DLs and MLs and most of the time they've been excellent. The only issue so far was an RPM extension that caused problems but that was experimental anyway and with a quick 30 minute rebuild everything was back up and running again with no rebuild of the VMs required. Couldn't have been any easier.
So maybe the reason life about Xen has gone quiet is because for some of us who are seen as early adopters of it just don't scream and shout. And yes I looked at VmWare. Its nice etc but its heavy! I don't want to lose huge performance on systems where I need it.
Come on Xen, we love ya baby!
Xen also suffers from a lack of good sales managers to communicate with Govt./corp. customers. Case in point: my organization is doing a massive data-center virtualization proof of concept. VMWare has provided evaluation licenses and great no-cost assistance. Xen has yet to return my call or e-mail me in response to any questions.
I've been using VMWare ESX server for a few years in production environments. This is primarily in corporate datacenters where server utilization is very low.
In our public facing co-lo facility server utilization is much higher and we can't justify the cost of ESX server. We can use xen with 2-4 virtual machines and reduce our cost and server footprint significantly.
The management tools are much better using virtual center compared to xen. The standard ESX interface isn't that much better then xen. Besides most production environments are more static so the management tools aren't that important.
I've been running VMWare Workstation since the 3.0 days on a variety of x86 hardware under RH7/8/9 and Fedora 1-7. I've found it runs OK, not great, with Win2K and WinXP guests.
At the office we just implemented 64-bit RHEL5 on new VT hardware with xen and RHEL4 guests. Performance has been excellent. Reliability has been excellent for the RHEL4 guests. I made the mistake of trying to install 32-bit WinXP OS under it. I know that 32-on-64 is not supported on RHEL5 until 5.1 comes out. But I tried it anyway, and it crashed RHEL5.
Documentation remains poor, but I "live and die" with Google so I was able to get all the answers I needed.
The support tools are terrible. VMWare makes it easy. I'm a Unix SA with 14 years experience, so I'm not at all averse to the command line, but I certainly appreciate the tools that VMWare has developed.
I remain hopeful that xen will take off. My just ordered desktop is a quad core with VT, so I am planning to install Fedora 7 with a WinXP fully virtualized guest. I would expect VMWare to run well on this hardware too, though officially VMWare is not supported at all. I always need the excellent patches available (thanks again google) to get it running.
All in all, I try to keep an open mind (no pun intended). I do like to run leading edge stuff, it's what floats my boat. I encountered a little resistance when I went with RHEL5/xen rather than VMWare for the virtual solution, so far it has no come back to bite.
bottom line: xen looks promising but it does have a long road ahead.
I wonder why people aren't mentioning the open source virtualization solution Virtualbox. It can run on Windows, Linux and the Mac and it supports many host operating systems unmodified, among them Windows. Since I found it, I am not looking at Xen anymore.
http://www.virtualbox.org/
The commenters have made a lot of good points and contributed a lot of information I didn't have when I wrote the post. It seems indisputable that Xen is doing well in ISP/hoster/Internet-facing environments. This is because it's obviously much cheaper than VMware, its performance at least for hosting Linux may be quite a bit better, and the immaturity of the management tools doesn't hurt as much as in the corporate data center. The Amazon EC3 and Oracle hosting examples, along with many others, support this view.
But it also seems indisputable at this point that Suse and especially Red Hat have dropped the ball as far as Xen integration is concerned. Maybe things will get better over time, but for now what the commercial Linux distros have delivered for Xen falls far short of the expectations they themselves created not so long ago.
Just last week, on the very same day I posted the piece, Red Hat CEO Matthew Szulik confessed to Wall Street that hardly any Red Hat customers have put Xen into production with RHEL5. His comments are pretty amazing:
I guess he's not talking to very many VMware customers, huh? Otherwise he might have noticed that quite a few of them are definitely "in production" and no longer think of virtualization as a "young technology". Another interesting point is that Red Hat has stopped saying the word "Xen" in public, all you get are generic references to "virtualization". There is a very interesting story behind this verbal quirk that I will try to cover in a future post.But in writing last week's post I probably should have drawn a sharper distinction between Xen and XenSource. The latter of course is the VC funded startup that employs many of the Xen project contributors, including its founder Ian Pratt. The day after I posted the piece, XenSource told Reuters it now has 500 customers, most of them running "just a handful of servers", aside from the big hoster examples like Amazon already mentioned. But having seen its sales boom in the last few quarters (presumably on the strength of XenEnterprise and its ability to host Windows Server as well as Linux), the startup now hopes to double or triple its installed base by the end of the year. That would certainly suggest that they are advancing faster up the adoption curve than VMware was at a similar point in its life cycle.
The big difference in XenSource's approach compared to the distros is that they are positioning their XenEnterprise product (which is a hybrid of open source Xen and their own closed source management tools) as a direct competitor to VMware ESX, something that Red Hat and Novell have not done with their more basic Xen offerings. Given the rapid rise of KVM, maybe Red Hat feels it doesn't have to play along with the Xen team to get a robust open source virtualization solution integrated into Linux. That may be so, but I have to wonder if Red Hat isn't underestimating customer demand for an OS-independent hypervisor that has serious management tools and does a good job with both Microsoft and open source stacks. VMware already has a solution like that, and XenSource is trying to put one together. Microsoft seems to be hoping that Novell and XenSource will help make it a serious contender in the dual stack game too.
In any case, in the run up to Longhorn/Viridian and on the eve of what is likely to be a sensational VMware IPO, XenSource is certainly worth a closer look.
I work for a communications company in the UK and we run a large majority of our network on Xen.
Due the the number of machines required to run our network and power allocation at our datacentres becoming increasingly scarce moving to a xen platform was a logical solution, which has proved itself as a stable and reliable environment.
Our original platform was based on Slackware with a Xen2 kernel and Slackware 10 guests.
The platform I have been involved in designing and rolling out across the network uses RHEL4 with a Xen3 kernel with the redhat patches pre-applied. I decided to go with fully virtualised Slackware 10 guests running on a Dell 2950 base utilsing the Xeon VT cores. During my testing of this platform I managed to get 12 guests building kernels running stably on the host, at 15 the host went non responsive however the guests were still building kernels!
To me Xen has proven itself to be stable and highly usable in the enterprise environment.
My only complaint is the lack of any kind of management tools that are bundled with XenSource: One of my colleagues had already written some management scripts for our Xen2 platform so I adapted them for the Xen3 platform. I also tried the RHEL5 management programs (which come out just after I had finished developing the RHEL4 based Xen3 platform!) but was not impressed. The best thing to do with Xen is to write some scripts tailored to your needs, because at the moment there is nothing flexible enough to meet everybody's requirements.
Xen still has some way to go to compete with the out-of-the-box usablity of vmware, but it is a very good solution for those willing to get stuck in.