A Chat with Carl Trieloff about Apache Qpid and Red Hat MRG

By Jeff Gould, CEO & Director of Research, Peerstone Research

In this three part series I discuss the emerging open source standard for business messaging, AMQP (Advanced Message Queuing Protocol). Part 1 provided a non-technical introduction to the basic ideas behind the protocol. In Parts 2 and 3, I chat with two leading AMQP developers, Carl Trieloff of Red Hat (Part 2) and Alexis Richardson of RabbitMQ (Part 3).

In addition to running Red Hat’s MRG initiative, Carl Trieloff is a key member of the Apache Qpid project.

Interview

Q: What is the status of the AMQP spec?

A : The spec is close to 1.0, it’s not quite there but it’s in process to get there later this year. It took a lot of work to get from 0.9 to 0.10, we needed to bring in additional reliability and make sure it could handle all the JMS semantics. We also optimized the layering of the stack, for example by separating levels of concern between hardware and software. The protocol is designed to make it easy for some low-level functionality to be handled in network hardware. Cisco is a member of the Working Group and they have played an important role here.

Q: Red Hat’s version of AMQP is based on the Apache Qpid project. Who is contributing to Qpid besides Red Hat?

A: As a rule the Apache contributor base is pretty anonymous. Currently Qpid has 19 committers plus 6 other significant contributors who are currently less active, plus 5 other guys who help run the project, plus 8 people who provide stuff to us and might eventually become members of the project. Certainly Red Hat is one of the larger driving factors behind Qpid, but we are not the majority by any means.

Q: What is the relationship between AMQP and JMS?

A : Our work is driven by business user requirements. JMS is widely used as an API for other MOMs like IBM and Tibco, so it’s a given that AMQP has to work well with it. The early versions of the spec weren’t rich enough to run JMS semantics, so we had to rework that. Now anything that does JMS is very easy to integrate with Qpid. One thing you have to remember is that JMS is just an API. Each underlying implementation of JMS is proprietary. The API allows you to abstract from the details of the particular implementation. IBM MQ and Tibco Rendezvous can both be accessed through JMS, but you shouldn’t expect them to interoperate. However with AMQP it’s different. The Qpid implementation will interoperate very well with the RabbitMQ and the iMatix implementations. You should also note that using JMS is an option for AMQP but not a requirement. So long as you’re not serializing Java objects you can talk JMS on one end and a different API on the other. Qpid has developed APIs for each of the non-Java versions of our AMQP client, including C++, C#, Python and Ruby. Some people have even asked for a CICS interface, we might do that.

Q: What is Red Hat’s strategy for bringing a commercial AMQP product to market?

A: Red Hat Messaging is a distribution of Apache Qpid. Qpid is to Red Hat Messaging what kernel.org is to RHEL, it’s the upstream repository. Red Hat adds additional components such as certain management features. Other companies have announced their own distros of Qpid [e.g. Iona]. Red Hat Messaging is also one piece of Red Hat MRG, which is our suite of messaging, realtime, and grid components, each with its own upstream repository, but all integrated together by Red Hat into a middleware product that sits on top of RHEL.

Q: Who is Red Hat MRG for?

A: Not every customer is necessarily interested in all three pieces. Some are, but not everyone. For example, there are customers with embedded applications who only use real time. This is a realtime kernel for RHEL that gives strong guarantees about the amount of time it takes to service requests, it’s very deterministic. There are other customers who want both realtime and messaging, i.e. AMQP. For example, the trading applications in the financial industry, and certain telecom applications. The grid component, which comes from the University of Wisconsin’s Condor project, is popular for HPC [high performance computing] and scale-out architectures. For example, a user could be running a spreadsheet locally and then execute a simulation remotely on a grid that resides in the cloud. So depending on the application, the user might be interested in all or only some of the MRG components. But MRG unifies things from a management perspective.

Q: Is AMQP going to be a serious threat to IBM or Tibco?

A : Probably AMQP won't cause people to rip out MQ, because the cost of reengineering existing running projects is too high unless there are additional factors. But we expect to see AMQP adoption on new deployments that offer better TCO or a different value chain, or where the customer wants the benefits of open source. There could also be cases where MQ gets updated to AMQP for business reasons, for example maybe if the user gets acquired or consolidated. We are going to compete with IBM on messaging, but we are certainly not waging war on them. We also partner with them for realtime Java.

Q: How will AMQP play with ESBs and SOA?

A: The role of an ESB [enterprise service bus] is translating message formats and protocols and doing work on messages, as opposed to actual transporting them from place to place. It’s essentially a place where you can plug in a bunch of services to work on your messages. The JBoss ESB, which is our platform for SOA [service oriented architecture], focuses on mediation components and provides the ability to link to multiple transports. But you still need an underlying messaging infrastructure when you deploy transport, and that’s what AMQP offers over and above the ESB itself. Our ESB currently supports JMS transport providers such as IBM MQ, and will support Apache Qpid later this year.

Q: What is the relationship between AMQP and Web Services ReliableMessaging (WS-RM)?

A: WS-RM came about because it was apparent that HTTP by itself was deficient as a transport protocol. So WS-RM aims to add reliability on top of HTTP. This is in a web services scenario where you are sending XML messages wrapped in SOAP envelopes. Some people who don’t like HTTP say send SOAP over JMS instead. Of course you can do the same thing with AMQP, it will work fine as a transport for SOAP and XML. It defines a complete wire protocol and therefore eliminates HTTP. But AMQP’s goal is broader than WS-RM. It’s intended for any kind of application-to-application message content, especially binary formats, and not just XML. It probably wouldn’t make a lot of sense to run WS-RM over AMQP, but someone could eventually do an RM-to-AMQP bridge. I think WS-RM had a lot of promise when it got started, but it has maybe lost a little momentum in the last few years.

Q: What guarantees that different AMQP implementations will actually interoperate?

A : There is no official mechanism as yet, but the AMQP Working Group has done a lot of work on interoperability. It’s true that at any given time different implementations will be at different points on the spec and therefore at different points on interop. But our experience has been that interop between the implementations is continually getting better, because the spec is getting better and less ambiguous, and because the developers are working very hard to track it. I think you need to have a very pragmatic approach to interop. If you look at Sun, they spend an enormous amount of money writing TCKs [Technology Compatibility Kit] for Java specs, but that doesn’t always guarantee interoperability.