Feature Article By Jeff Gould
Copyright © 2007, Peerstone Research Inc, All rights reserved.

March 30, 2007
The server operating system wars never seem to slow down. Last week it was Red Hat's turn with the announcement of Red Hat Enterprise Linux 5 (REHL5), which incorporates the Xen open source hypervisor. Naturally there's also the endless market speculation about the final feature set and likely arrival date of Windows Server 2007 (aka Longhorn). And then there's Solaris, which with its nice value-add features like DTrace and its new status as open source software is making something of a comeback, it seems.

In this industry we all spend a lot of time following the constant tit-for-tat among the vendors. Frankly it can be a lot more exciting than some other spectator sports I know of, like politics. But every time I start to dig into all the white papers and benchmarks and analyst reports to try to figure out which of these fine offerings has finally managed to get a leg up on the competition thanks to its latest bell or whistle, my head starts to spin. Is it really possible to keep up with all this stuff? Is someone ever going to win the operating system wars and put an end to our confusion? Or are we just on an endless treadmill of tweaks and improvements and competitive breakthroughs that will never settle down into a steady state? And so doubt creeps over me, and I begin to wonder if we haven't all fallen into the software industry equivalent of Groundhog Day.

Then a light goes on, I slap myself on the forehead, and I say to myself, "It's not about the operating system, it's the stack, stupid!".

In other words, people don't really sit around wondering which operating system they should use. Instead, they decide which applications they want to run, and the OS usually picks itself. At least, that's the thought that came to me while reading the interview with Rich Green that Sun posted on their web site last month. Now Green, who returned to Sun last year as Exec VP for Software, is one of the smartest people in software, so he's always worth listening to (full disclosure: I did a long Q&A with him a while back when he was still an EVP at Bill Coleman's datacenter automation startup Cassatt).

It will come as no surprise that Green's pitch is all about how Linux users should come back to Solaris. He has a lot of good points to make, but his basic argument is that Solaris is a lot better than it used to be at supporting the web tier applications that have been the driving force behind the rise of Linux in the datacenter.

"Solaris has always been superb on the back end. But to be honest with you, until a couple of years ago, the Solaris Operating System did not fulfill all the requirements of the edge and Web tiers. Today, however, Solaris has been optimized for the one-, two-, and four-way microprocessors used in AMD and Intel chipsets for many of the same servers that run Linux today. This means that the operating system provides an entire solution for both the database and presentation tiers." I'm only half convinced by this. I mean, I don't doubt that one of the new Sun servers packed with dual or (soon) quad core chips from Intel or AMD – or even one of Sun's own eight core Niagara chips – makes a nifty replacement for the rack of Dell Xeon boxes at work in a typical web tier. But I'm not so sure that masses of Apache, PHP and MySQL users who are today happily chugging along on Linux are going to start lining up at Sun's door any time soon. In fact, Red Hat seems to have seen this threat coming. Their just announced Red Hat Exchange will be an online store for commercial open source software. Although plenty of people will continue to run their open source stacks on free versions of Linux rather than RHEL, the coming of RHX is bound to have a big impact on the Linux commercial ecosystem.

As for the Oracle users who were thinking of moving from Solaris on Sparc to Linux on Intel-AMD (can we just call it "Amtel"?), well that's another story. There's a lot of anecdotal evidence that Oracle on Solaris continues to outperform Oracle on Linux in high-volume transaction processing applications. This has been one of the historical sweet spots for the vendor Unix systems. And it's probably no accident. It's widely known in the industry that database code has to be carefully optimized to a specific OS and multiprocessor server platform if you want to maximize performance. Most modern operating systems contain special APIs to make this optimization possible. Over the years Sun and HP engineers have spent a lot of time in Oracle's lab carefully tweaking the Oracle database kernel to make it scale on their big SMP Unix boxes. But as far as I know, no one is doing this work for Linux. (Want a geeky example? An industry source recently confided to me that when the server hardware vendors run their Oracle validation test suites on Linux they don't even bother to turn on the non-uniform memory [NUMA] API in the Linux kernel.)

But to really take the measure of Green's claim that Solaris is winning back lost ground to Linux, we need to look at their respective installed bases. According to IDC, there were about 1.5 million active Solaris servers installed around the world at the end of 2005. According to Sun's web site the vendor shipped about 350,000 servers last year, some of which must have been replacement machines rather than new installs. So lets say the Solaris installed base has grown to 1.75 million servers by now.

How big is the Linux installed base in comparison? I haven't seen any published IDC or Gartner numbers on this, but there is a fascinating slide on Microsoft's web site that breaks down the worldwide server installed base into 13 segments and gives an estimate of Linux's share in each one. According to Microsoft, there were 24 million servers in use in 2005, and 15% were running Linux. If we put the server installed base at 27 million today, and then bump up the Linux share to 17%, we get a total Linux server installed base of around 4.6 million machines, or roughly three times the size of Solaris. However, one could object that this is an apples-to-oranges comparison in the sense that most Solaris is in the datacenter while a lot of Linux is out on the perimeter. IBM has recently estimated the number of Linux servers actually installed in datacenter environments at 1.6 million worldwide, which puts Linux slightly behind Solaris.

Conclusion: Solaris is holding its own in the datacenter applications like database, but is still far behind Linux as a platform for the broader open source software stack exemplified by apps like Apache, PHP, Tomcat, or Sendmail. If Solaris is going to live up to Rich Green's claims for it, it will have to break out of its traditional high-end comfort zone. But given the vast proliferation of free Linuxes out there, this is quite frankly going to be hard to do, at least as a revenue generating proposition for Sun.

Free Linux is one thing, but what about commercial Linux? The continued strength of Solaris in the datacenter makes it a significant impediment to Red Hat's effort to migrate up the stack from the barebones operating system layer to the rich application layers above it. These layers include things like virtualization, clustering, security and management. You see, Red Hat is no longer really synonymous with everyman's Linux. Rather, it has evolved into a kind of premium "rich man's" variant of Linux whose true target is a relatively small number of large and wealthy IT organizations. Think Wall Street bulge bracket brokerage firms, major banks, tier one telcos, and big Federal agencies. These are the customers who are going to pay $2,499 per server per year for 24x7 support on RHEL5 Advanced Platform, or even more for Red Hat's new Datacenter Solutions offering.

But the problem for Red Hat is that to win over the premium segment it will have to go head-to-head with Solaris (and, to a lesser extent, with IBM's AIX and HP's HP-UX). Sun has at long last figured out that it was pricing itself out of the market, in effect creating a pricing umbrella that allowed interlopers like Red Hat and Microsoft into the datacenter game. Now Sun has slashed its software support prices and revamped its hardware architectures. It still isn't cheap, not if you buy the whole Sun stack and the support that goes with it, which is why I suspect the company will never catch up with Linux in the broader server market. But it is in a much stronger position to hold its own in the premium market segments that really count for its bottom line. And that spells trouble for Red Hat.

While Solaris and RHEL duke it out for control of the premium "operating systems derived from Unix" segment, both players are overlooking the threat from Microsoft. According to IDC, Windows Server commands about two thirds of total server unit shipments and drives almost three times the dollar value in server system sales as Linux ($5.3 billion vs. $1.8 billion in Q5 2006). It's pretty hard to find a datacenter these days that doesn't have some Windows Server boxes lurking somewhere.

Although the open source world scarcely deigns to notice it, with SQL Server and Exchange in particular Microsoft has assembled two of the most widely used enterprise applications in the market. The power of these applications to attract and retain customers for the Microsoft stack is far more important than most Linux fans recognize. It's hard to get exact numbers, but some insiders put the total SQL Server installed base at more than one million. Compare that to Microsoft's estimate (on the same slide cited above) that the total world installed base for database servers is three million machines, or Oracle's statement in its marketing materials that it has around 250,000 customers. Of course Oracle's database business is larger in dollar terms than Microsoft's, because its database is a lot more expensive. But SQL Server 2005 is growing sales at over 30% more than a year after its release, while Oracle's database revenue in its latest quarter was close to flat.

The danger that commercial Linux faces today is that its leading proponent, Red Hat, has forgotten to pursue a volume strategy and has painted itself into a premium-priced corner that it is busy fighting over with Sun. Fedora is the volume strategy you say? Well, yes, Fedora is free-as-in-beer, which makes it a lot cheaper than RHEL. But that doesn't really address the problem. You see, it really is all about the stack. Linux still owns the web tier, and it doesn't look like anyone can take that away in the foreseeable future. But MySQL and Postgres aren't viable replacements for Oracle or SQL Server, and PHP and Ruby aren't viable replacements for enterprise Java or .Net. And nobody seems to be doing the work needed to make Linux scale on the new multiprocessor scale-up server hardware that is coming back into vogue. It really is beginning to look like Linux is stalling in the datacenter while Solaris holds its own and Microsoft races ahead.

Perhaps its time that the Linux community stopped obsessing about the arcana of intellectual property rules and started worrying more about what customers are actually buying. Remember, it's the stack, stupid!.