By Jeff Gould, CEO & Director of Research, Peerstone Research

The suits at IBM are in a snit. According to the Wall Street Journal, the world’s largest computer company is threatening to pull out of certain international standards bodies because it has “become frustrated” by what it views as their “opaque processes and poor decision-making.” IBM’s press release on the subject is somewhat milder in tone, being a tree-hugging, planet-loving paean to the virtues of open standards and, just as important, open standards making processes. Who could object to that? However, beneath the velvet glove there lies a fist of some decidedly more ferrous material. IBM wants everyone to know that if it doesn’t like the quality of certain standards bodies’ work, it will lobby for changes in the way they operate, incite local governments to force them to reform, or even drop out of them altogether.

My goodness. Whatever could account for such a tantrum? IBM’s press release coyly refuses to name names. But IBM execs apparently were less timid in their off-the-record comments to the Journal’s reporter, who attributes the move to anger in Armonk over Microsoft’s hard-fought and bitterly controversial but ultimately successful effort to get its Office Open XML (OOXML) file format endorsed by the International Organization for Standardization (ISO). IBM blames the ISO and especially its member standards organization Ecma International for resisting its furious lobbying effort to keep the Microsoft standard out in the cold.

So if the real reason for IBM’s sortie against insubordinate standards bodies is OOXML, then the obvious question that arises is: what the heck is OOXML, and why should anyone care about it? To answer this question requires a brief segue into the arcane of office document file formats. Although some may find this digression more boring than a Joe Biden speech, I promise you it will be much shorter, and hopefully free of gaffes. The nub of the matter is that the file formats of common PC apps such as word processors and spreadsheets, which have long consisted of obscure and proprietary binary encodings, are now at long last engaged in a historic migration to XML. This migration will bring many benefits to users. In particular, XML file formats will make it feasible for tool vendors and enterprises to manipulate office productivity files programmatically. Organizations that embed strategic value in their documents will be able to do all sorts of neat new things with these formats, such as dynamic assembly of complex documents, systematic content reuse, content searching with XQuery, automatic verification and updating of documents, intelligent archiving, and so on.

Major shifts in technology often disrupt established vendors and create opportunities for challengers. By the early 2000s both Microsoft and its leading enterprise IT competitors such as IBM and Sun realized that the transition from binary to XML had the potential to stir things up in the office suite market. Naturally IBM and Sun saw this as a wonderful opportunity to put a dent in Microsoft’s multi-billion dollar Office franchise while promoting their also-ran OpenOffice suite. Microsoft on the other hand had a somewhat more jaundiced view of the matter. For Redmond, preserving and protecting the Office cash gusher was a matter of corporate life or death. But both sides recognized that XML was inevitable, and set to work developing their own XML file formats.

IBM, long-schooled in the ways of international bureaucracies, soon realized that the OpenOffice XML format, ODF, could be much more than just a snazzy new feature in the product. If Big Blue could only get ODF approved as an official international standard, preferably by some august institution such as the ISO, then it could persuade cash-strapped but fashion-conscious government agencies and universities that they could be virtuous and save money at the same time. All they had to do was ditch the proprietary formats of the evil empire in Redmond for the hip and crunchy new “open standard” ODF. Never mind that OpenOffice had only a tiny fraction of the installed base of Office. Never mind that it had a far more limited feature set. Never mind that Microsoft would eventually prove itself willing to cut special deals with developing countries (for example, selling Office seats to the government of China for $10 a pop). No, the noble purity of an authentic “standard” would prevail over such vulgar commercial and technical considerations. At least that was the plan.

Initially Microsoft ignored the threat from ODF because it assumed that its overwhelming market share would turn whatever XML format it developed into a de facto standard. But in 2005 ODF became a de jure ISO standard, and certain government agencies in Europe and America soon suggested that they would mandate its use (one early win for ODF was Massachusetts, though the state later changed its mind). Waking up to the danger, Microsoft quickly took its OOXML format to a private European standards body, Ecma International, of which both IBM and Microsoft were long-standing members. A rash of vicious bureaucratic infighting soon broke out, the details of which are too numerous and too tedious to recount here. But to make a long story short, in April of this year, after several reversals, Microsoft succeeded in getting ISO to ratify OOXML. Customers could now choose from not one but two official international standards for XML office formats.

The IBMers went berserk. They were beside themselves with rage. One IBM employee even blogged about how Microsoft had “raped” the ISO. IBM’s open source allies also got into the act (though not all of them). Some made quite cogent criticisms of the “disharmony” and “engineering horror” of OOXML, for example in this article by IBM ODF expert Rob Weir. But others quickly descended into creepy, juvenile insults (scroll down to the reader comments in Weir’s article).

Now it is true that at more than 6,000 pages the OOXML spec has all the lean elegance of an Airbus A380 wiring diagram. In other words, it’s a monstrosity. If it had the misfortune to be someone’s baby, even the parents would call it ugly as sin. But OOXML got that way for a reason, namely, the obvious user-driven need to permit the migration to XML – without undue loss of function – of all those billions of older binary-based Office documents that populate the world’s hard drives. This was a very complicated problem, and if you seriously believe that a simple solution was feasible, well then I have a couple of bridges here in San Francisco I’d like to sell you (hey, subprime financing available, guaranteed by Uncle Sam).

The OOXML critics aren’t wrong to lambaste OOXML’s complexity. In an ideal world we would start all over from scratch. I personally would like nothing better than to escape from my decade-long entanglement with Microsoft Office. I’ve never been a fan of the new versions of Word and Excel in Office 2007. But the reality is that I can’t get rid of Office, because those swaggering defenders of customer freedom over at OpenOffice.org haven’t seen fit to satisfy my most basic requirements, such as compatibility with Word’s indispensable outline mode. Until they do, OpenOffice will remain a little-used 330 megabyte blob on my hard disk. Notice that this is not even a file format issue, but an application functionality issue. As a user I don’t care about the format war between OOXML and ODF. No amount of holier-than-thou preaching will convince me to prefer one over the other. I only care about functionality, convenience and price. And I’ll bet that at least 9 out of 10 office suite users on the planet agree with me (I’m being very conservative here, the true figure is probably more like 99 out of 100).

All standards are a compromise between the interests of the sellers and the buyers. The sellers agree to play because they believe they will sell more with the standard than without it. The buyers agree to play if – and only if – the standardized product does whatever it does cheaper and perhaps even better than the proprietary alternatives. There is nothing intrinsically noble or magical about standards. They are merely pragmatic business arrangements that exist because they create tangible value for the stakeholders, who are motivated only by self-interest.

There are many different kinds of standards. Some are purely de facto. They emerge spontaneously from products that have by hook or by crook achieved overwhelming market dominance. Obvious examples include Intel’s x86 processor family, Microsoft’s Office suite, Adobe’s PDF, and IBM’s mainframe architecture. Others standards bubble up in unplanned but serendipitous ways from the user community. This was the case of the TCP/IP networking protocol, the Unix and Linux operating systems, and the World Wide Web’s HTTP and HTML. Still other standards, such as ODF, Java or the WS-* standards for Web Services, are birthed top-down in cumbersome, contentious vendor committees. Standards of this last variety are unfortunately the most common, because they are the easiest to create. But they are also the least likely to win broad acceptance among users. Some do and some don’t. Java and of course XML are spectacular examples of successful vendor-driven standards. The most one can charitably say about ODF, on the other hand, is that the jury is still out, and is not expected back anytime soon. Vendor-driven standards are frequently legitimized by official standards bodies, but they are rarely of lasting influence unless they achieve significant market uptake (one rare exception to this rule is that most influential of flops, the seven-layer OSI network stack).

If a product has 500 million users, then it is a de facto standard, and the only serious choice a standards body has is to negotiate whatever improvements it can (as the ISO did when it deprecated some of the old Microsoft binary cruft in the Ecma version of OOXML) and then get the hell out of the way. Real standards are not made in heaven or in standards bodies. They are made in the hurly-burly of the marketplace, whose motive forces are greed, ingenuity and luck. Like the human genome, they are not optimally elegant clean-sheet designs, but messy kludges that carry within them the convoluted details of their prior history. But, honed by the relentless pressure of natural selection (aka customer choice), they possess an attribute that most top-down built-by-committee standards lack, namely the ability to do the work that users actually want done.

While IBM’s press release hints that it might drop out of standards bodies such as Ecma that have earned its displeasure, Big Blue standards blogger Bob Sutor quickly discounted this possibility. I take him at his word. If you read the press release carefully, it becomes apparent that the real plan is not to withdraw from these organizations but to take them back by force. IBM is going to the mattresses and preparing to wage trench warfare in the world’s standards making bodies. IBM says it opposes the participation of “proxies or surrogates” in standards creation and approval. But in the very same breath it vows to “encourage diverse participation” in standards bodies. Translation: it’s going to push harder than ever to pack the local bodies with its own proxies and surrogates in order to cut Microsoft off at the knees the next time around. IBM’s document is a thinly-veiled invitation to governments of developing countries to join in the witch-hunt high-minded effort to bar “politically incorrect” vendors from government contracts by means of selective, agenda-driven standardization. In other words, if you can’t win in the marketplace, call the politicians.

Standards bodies are – always have been, always will be – rife with competitive horse trading, hidden agendas and outright obstructionism. ODF is an attempt by its backers to win in the standards bodies what they failed to win in the marketplace. OOXML is an attempt by Microsoft to leverage the very same standards bodies to keep what it won. This is not a pretty sight, but when we visit a sausage factory we shouldn’t be surprised at what goes on there. We can do ourselves all a favor by dropping the cant and the hypocrisy exemplified in IBM’s latest attempt to “purify” the standards making process. Let’s stop pretending that this whole process is or ever could be governed by anything other than self-interest. But let’s keep in mind that the self-interest of those greedy sellers is always counterbalanced by the self-interest of we the buyers. In the immortal words of Adam Smith, “It is not from the benevolence of the butcher, the brewer or the baker, that we expect our dinner, but from their regard to their own self interest.” If we really want the best standards, we should stand back and let them bubble up spontaneously from the market.