Tuesday, November 30, 2010

Camel Integration with WebSphere MQ

This post will describe my experience with implementing a Camel route that integrates with WebSphere MQ. The Camel route runs inside Servicemix as an OSGI bundle. I used the Enterprise version of Servicemix from FuseSource available here:

This use-case was developed to show how to achieve interoperability with WebSphere MQ.
You can download the example code here .

You can extract this file and, using Eclipse, import as an "existing Maven project". You will need to have the M2Eclipse plugin installed.


The following diagram shows what is happening in the use case created to demonstrate interoperability with WebSphere MQ

The use case consists of a Camel route (deployed inside Servicemix) that receives an HTTP message from an external client. The arrival of the HTTP message triggers the route. The route is pretty simple. It simply takes the payload of the HTTP message and drops it onto a Websphere MQ queue.

The HTTP message received (that triggers the route) is a two-way (request/response.) The message exchange pattern (MEP) for any route is generally determined by the first (consumer) endpoint in a route. Therefore the MEP for this particular route is request/response.

In this route, the JMS producer is responsible for placing the message on the pre-configured MQ queue. Because the route is request/response, the JMS producer will by default wait on a response to arrive on the response queue. This response queue is determined by whatever queue name is set in the outgoing JMS message’s JMSReplyTo header. The JMSReplyTo header is a standard-based JMS message header that indicates which address (queue) the JMS consumer that receives the message, should reply to.

In some cases the replyTo queue could be a temporary queue set up (per-message) by Camel on the fly to listen for reply (dynamic replyTo). In other cases, the replyTo queue will be a well-known location (static replyTo.) In the current use-case the replyTo queue is a well-known (static) location. This follows a pattern common to many JMS applications.


The following shows the Camel code for this route.


.to(targetUri + "?replyTo=SYSTEM.SOAP.RESPONSE.QUEUE");

The code simply listens for incoming HTTP messages (via the camel Jetty component) and forwards the payload to a Websphere MQ queue. The route sets the value of the “JMSreplyTo” header using an option on the outgoing targetUri. Thus the route, after sending to the targetUri queue, will wait on a response on the queue specified in the ‘replyTo’ option. In this case the route will listen for a response on “SYSTEM.SOAP.RESPONSE.QUEUE”. Camel will by default use a message selector to identify the response message, searching for the response message that has a corrrelationId that matching the messageId of the original message. In this way, Camel handles the message correlation for you without additional code. The response then gets converted and sent back to the original client via HTTP.

The Spring configuration for WebSphere MQ, used in above use case is shown below:
<beans xmlns="http://www.springframework.org/schema/beans"

<camelContext xmlns="http://camel.apache.org/schema/spring">
<routeBuilder ref="mqRoute"/>

<bean id="mqRoute" class="com.fusesource.camel.MqRoute">
<!-- This is the "request" queue -->
<property name="outgoingQueue" value="outgoingPayments"/>
<!-- This is the "response" queue -->
<property name="incomingQueue" value="incomingPayments"/>


<bean id="myTransform" class="com.fusesource.camel.MyTransform"/>

<bean id="wmq" class="org.apache.camel.component.jms.JmsComponent">
<property name="connectionFactory">
<bean class="com.ibm.mq.jms.MQConnectionFactory">
<property name="transportType" value="1"/>
<property name="hostName" value="localhost"/>
<property name="port" value="1414"/>
<property name="queueManager" value="QM_TEST"/>

Building the example
First make sure you have built the example and installed into local maven repo:

mvn install -DskipTests=true

You can see the routing rules by looking at the java code in the src/main/java directory. The Spring XML configuration lives in src/main/resources/META-INF/spring/camel-context.xml

WebSphere MQ Dependencies

When implementing a Camel solution to interoperate with WebSphere MQ, there is a required set of IBM-specific libraries needed. For the purpose of this POC, these libraries were obtained by downloading a WebSphere MQ v. trial for Windows.

Making Dependency Available to Runtime

A set of MQ libraries are required to be installed into FUSE ESB (Servicemix) making them available at run time. This set of dependent runtime JARs is provided as part of a MQ installation. The libraries, and the folder from and MQ installation where they can be found is shown below:


These JARs packaged as standard OSGI libraries by IBM and therefore recognizable and deployable into any standard OSGI container. These libraries can be deployed into Servicemix easily using the following commands:

osgi:install -s file:C:/Progra~1/IBM/WebSph~1/java/lib/OSGi/com.ibm.msg.client.osgi.jms.prereq_7.0.1.3.jar
osgi:install -s file:C:/Progra~1/IBM/WebSph~1/java/lib/OSGi/com.ibm.msg.client.osgi.jms_7.0.1.3.jar
osgi:install -s file:C:/Progra~1/IBM/WebSph~1/java/lib/OSGi/com.ibm.msg.client.osgi.wmq.prereq_7.0.1.3.jar
osgi:install -s file:C:/Progra~1/IBM/WebSph~1/java/lib/OSGi/com.ibm.msg.client.osgi.wmq_7.0.1.3.jar
osgi:install -s file:C:/Progra~1/IBM/WebSph~1/java/lib/OSGi/com.ibm.msg.client.osgi.nls_7.0.1.3.jar
osgi:install -s file:C:/Progra~1/IBM/WebSph~1/java/lib/OSGi/com.ibm.msg.client.osgi.commonservices.j2se_7.0.1.3.jar
osgi:install -s file:C:/Progra~1/IBM/WebSph~1/java/lib/OSGi/com.ibm.mq.osgi.directip_7.0.1.3.jar
osgi:install -s file:C:/Progra~1/IBM/WebSph~1/java/lib/OSGi/com.ibm.msg.client.osgi.wmq.nls_7.0.1.3.jar

All but the last bundle are deployed with the ‘-s’ option (meaning to auto-start the bundle). The last bundle above is a bundle “fragment” so no start is required.

As a convenience, an optional step is to install these IBM MQ OSGI bundles into your local maven repo (or to a corporate Nexus repository.) If you do this, you would be able to use the Servicemix "mvn:" prefix with osgi:install (rather than "file:") telling Servicemix to use the local Maven settings to find the bundle and install it into the container. First it tries to find "mvn:" prefixed artifacts in the local Maven repository, it then tries any external configured repository.

Other Dependencies
There are a few features you need to make sure are installed into the Servicemix container prior
to deploying the sample.

features:install camel-jetty
features:install camel-jms
features:install camel-spring

Deploying the sample into Servicemix

Once all the dependent runtime MQ bundles are installed into Servicemix, you can install
the example bundle into Servicemix using the following from the servicemix command console:

osgi:install -s mvn:com.fusesource.examples/camel-mq-osgi/1.1

To enable logging for this example type:
log:set DEBUG com.fusesource.camel
For an even higher level of logging (logging Camel specifics)
log:set DEBUG org.apache.camel

Running outside of Servicemix

As an alternative to running this route in Servicemix, you can also run the camel route
in standalone mode outside of Servicemix using the camel maven plugin (see pom). This allows you to start the route using the "mvn camel:run" command.
When you run example using 'mvn camel:run' the camel-maven-plugin will construct a classpath from any Maven dependencies listed in the POM with scope "compile." So in order for maven to find these runtime dependencies, you will need to install this minimal set of IBM runtime libraries in your local maven repository. Similar to the IBM MQ OSGI runtime libraries, these jars can also be found in a installation of IBM MQ. You can install these JARS into your local maven repo as follows (the following assumes you have installed Websphere MQ version 7.)

mvn install:install-file -Dfile=C:\PROGRA~1\IBM\WEBSPH~1\java\lib\com.ibm.mqjms.jar -DgroupId=com.ibm -DartifactId=com.ibm.mqjms -Dversion= -Dpackaging=jar
mvn install:install-file -Dfile=C:\PROGRA~1\IBM\WEBSPH~1\java\lib\dhbcore.jar -DgroupId=com.ibm.mq.dhbcore -DartifactId=dhbcore -Dversion= -Dpackaging=jar
mvn install:install-file -Dfile=C:\PROGRA~1\IBM\WEBSPH~1\java\lib\com.ibm.mq.jmqi.jar -DgroupId=com.ibm.mq -DartifactId=com.ibm.mq.jmqi -Dversion= -Dpackaging=jar

It was necessary to take special action to install these artifacts into a well-known location (local Maven Repo) and tag the artifacts with appropriate naming scheme. Note the values specified for groupId/artifactIds are very sensitive to typos (this was the cause of some time spent debugging during testing - so be careful.) I took the liberty to define groupId and artifactId for the Maven artifacts associated with these IBM libraries. These names were based on common Maven conventions. It is unknown whether IBM maintains an appropriate public Maven repository that contains such “official” artifacts but none could be found when implementing the POC. If it is determined that such a public repo exists, then this step might be avoided.

Once the libraries are added to the local Maven repository as shown above, then the ‘pom.xml’ file for the given project can include the dependencies, using the groupId, artifactId, and version in order to identify the dependency to Maven. Maven should then be able to find the dependency in the local repo.

As a convenience, an optional step would be to install these jars into your local maven repo (or to a corporate Nexus repository.) In which case you could retrieve the dependencies from there and manually installing the JARS to the local maven repo would not be necessary.

Testing the Route
Whether running inside Servicemix or in standlone mode, the technique for triggering the route will be the same. To trigger the route you can perform an HTTP POST at the URL "http://localhost:8888/placeorder", sending in any valid XML as the content of the message. (I use a simple tool like SOAPUI or CURL to send http requests) The route will take the content of the incoming HTTP post and put it on the appropriate queue (as specified in the "camel-context.xml".)

As a convenience for testing, a secondary "server simulator" route has been added (see "MqRoute.java") that will listen on the outgoing queue. The route that is listening on this queue will pick up the message and place a response on the designated response queue (copying messageId into correlationId) per JMS request/reply convention. The primary route is then able to receive the response on expected queue and correlate it with the request.