CXF Transformations Revisited: Usage within a Camel Route

The CXF transformation feature provides a nice way to declaratively specify certain simple transformations – e.g. replace element names, add elements, change namespaces, etc. My previous blog post described this mechanism in detail.
This feature makes it easy to describe the transformation in a map in Spring XML. The ease-of- use in declaring a transformation with XML makes it a very appealing solution. Such an easy, declarative mechanism for performing transformations is not available in Camel.

The out-of-the-box use of CXF transformation feature in a Camel route is restrictive because the transformation is tied to the CXF endpoint, so the transformations defined cannot be applied to messages at any arbitrary location within a route’s pipeline. 

The purpose of this investigation was to determine whether the API that implements the CXF transformation feature could be de-coupled from CXF and re-used from within a custom Camel processor to perform transformations on messages at arbitrary locations within a route.

A prototype was written that proves it is indeed possible to use the CXF transformation feature to define transformation mappings (via XML) using the same syntax as the CXF feature. The transformation mapping is assigned to a generic processing bean. The processing bean can then be inserted at any location within your camel route.
Implementation
To implement this pattern, you will define the transformation in your Spring XML (‘camel-context.xml’) using the transformation mapping syntax. The mapping language is the same as defined here:

http://cxf.apache.org/docs/transformationfeature.html

An example of how this would look in Spring is shown below:

This bean declared above is an instance of the class “gov.foo.services.enterprise.deqy.transformer.TransformFeature”. This java class has several properties (expressed as maps) that will hold the logic required for the transformation.

The following code snippet lists the implementation of the underlying java class represented by the bean declared above.

The “process()” method (line 20) will get called as the message passes through the bean. The code will pull out the body of the message as an XMLStreamReader (line 22-23). Notice line 27 calls the “org.apache.cxf.staxutils.transform.InTransformReader” constructor with the appropriate maps.  This is where the code re-uses the CXF transformation implementation. The XMLStreamReader that is returned has been transformed by the CXF API. This is then set as the body of the message that gets passed along in the route. 

The following Camel route ‘camel-context.xml’ expressed in Spring XML, puts it all together and shows how the transformation can be placed into an arbitrary location in any route. This route will pick up a file from ‘inbox’ directory, pass along the content of the file, which is sent to the processing bean that performs the transformation defined in the same Spring file (shown by the mapping code shown previously.)  The content of the transformed message will then go to another file component that will write the content to a file in the outbox ‘directory’.

The above Camel route ‘camel-context.xml’ expressed in Spring XML, puts it all together and shows how the transformation can be placed into an arbitrary location in any route. This route (lines 54-58) contains the processing bean (line 56) that performs the transformation.

Overview of CXF Transformation Feature

The CXF Transformation feature allows you to declaratively define a transformation to change namespaces or append/change/drop elements. The transformation is very much tied to the CXFEndpoint.

Intended Use-cases 

The use-cases for which the CXF transformation feature works well are the following use-cases.

1. Transforming Incoming Requests from clients - A route exposes an interface to clients via a WSDL.  Some clients may not be capable of sending messages (or receive response) that adhere to the interface defined by the route’s WSDL. The CXF transform feature will allow you to define transformations on the CXFEndpoint so that incoming non-conforming messages from clients can be transformed and “fixed” to adhere to the expected interface defined by the WSDL. This allows clients to send non-conforming messages to the route (e.g. incorrect namespace or element names) – messages that would otherwise cause an error when sent to the route - and you can define the appropriate transformation on the CXFEndpoint to “fix” the incoming requests to conform to WSDL, and likewise modify outgoing (well-formed) response to something client expects.
2.  Transforming Outgoing Requests to backend - A route sends requests to backend service that are well-formed according to a specific WSDL, but the backend service expects some tweak to what the route is sending. This could apply for example, in the case where the route’s CXFEndpoint (client) is ahead of the backend service on changes to element names or namespaces that adhere to an enterprise standard. At some point the backend will be updated to accept these changes. However until the backend service is conformant, the transformation applied to the CXFEndpoint (client) in the route will tweak the outgoing messages so that they can be understood by the backend service, and likewise the responses from the backend service can be transformed as needed into the well-formed format according to the enterprise WSDL.

Transforming Incoming Requests from clients

Let's say we want our route to expose the interface defined by a new "enterprise" WSDL, and we want to go ahead and deploy that. Even if there are clients that are not ready to adhere to the new WSDL, clients can continue to send requests that adhere to the "old" WSDL (e.g. operation names, different namespace, different element names, etc.)  Existing clients can continue to send requests containing non-conforming messages.

In our new route, we just need to include a mapping in the spring file that defines the transformation on incoming messages. So as long as we add the transformation rules into our route, it is able continue to service clients that don't necessarily adhere to the new WSDL, and (if needed) the route will modify the messages as defined in the transformation.

Below shows the Spring configuration a route that exposes the interface defined in ‘deqy.wsdl’.  Imagine this is our “new” interface. If there are clients that must continue to call our route that can’t change to the new WSDL, then the route can be configured (via the Transformation mapping) to accommodate those existing clients.

The above code snippet is a ‘camel-context.xml’ (Spring configuration) for a Camel route that listens on a CXF endpoint. The CXF endpoint points to a WSDL that defines the route’s interface. The route includes a CXF transformation map that declaratively defines mappings that should be done to non-conforming messages as they enter the route. Note the mappings are defined in such a way that they won’t affect messages that do adhere to the WSDL.

The above Spring defines a route starting at (line 43) that begins with a CXF endpoint (listener). The CXF endpoint is defined at lines 32-38 with the transform feature being configured starting at line 35.  The individual transformations are defined at lines (20-29.)

As you can see it is pretty straightforward to configure a route to accommodate for payload transformation to be applied to incoming messages. Likewise it is just as easy to apply a transformation for the corresponding response back to client that will convert the compliant response from the route into a (potentially non-conformant) response that the client expects.

Transforming Outgoing Requests to backend

For a scenario where a route sends a request to a backend server, you may have written your route CXFEndpoint (client) that adheres to a particular WSDL but the backend service for some reason or another needs the message to be tweaked slightly.  To apply a transformation to a request message before it is written to the wire and a response before it gets dispatched to the route you can use an outgoing transform.

The following code shows the implementation of an outgoing transform feature.

Lines 29-42 define the transform mapping for ‘outTranformElements’ – the mapping on requests as they are sent from route to the backend service, followed by ‘inTransformElements’ – the mapping on response before the message re-enters the route.

See lines 44-51 for the CXFEndpoint ‘deqyService’, where the transform feature is applied (lines 48-50).

The route starts at line 54. The route reads a message from a file in ‘inbox’ directory, and sends it to the backend service. The route receives the response, and writes response to a file in ‘outbox’ directory.

Consider a test where the message content of the file dropped into ‘inbox’ conforms to the  WSDL defined on the outgoing CXFEndpoint – i.e. ‘deqy.wsdl’.    Imagine the backend service requires a tweak to a well-structured request. For example the namespace the backend service expects in incoming request and writes to response is different from what is defined in the WSDL. We can apply any such tweak required by backend service using a transform defined on the endpoint.  In this scenario the well-formed message we drop into inbox will get transformed just before it is sent to the backend service based on the transform defined in ‘outTranformElements’. Likewise, the response from backend service gets transformed based on the ‘inTransformElements’ before the response message re-enters the route.

A Workable Solution using CXF transform Feature

The following diagram shows how a message flows through a route that consists of incoming and outgoing CXF Endpoints. Both incoming and outgoing CXF endpoints have transforms defined.  The location in the flow where the transforms are executed is shown in the diagram.

Consider a client that is written based on the interface defined by Service ‘A’ WSDL.  The client sends a request to the route using the format based on Service ‘A’ WSDL. The incoming ‘inTransform’ (see position 1 in diagram) will convert messages from the (non-conforming) format sent by clients, into a format adhering to the Service ‘B’ WSDL. Without this transform, the incorrectly formatted message sent from client would cause an operation/binding mis-match when it hit the route’s CXFEndpoint that begins the route, and this would cause the request to fail on entry to the route.

The message continues through the route, through Processor A and B and then reaches the consumer CXFEndpoint. This CXFEndpoint will call the backend Service ‘B’. The format of the message is correct (adheres to Service ‘B’ WSDL) and Service ‘B’ will respond with correctly formatted message. In this solution, the ‘outTransform’ and ‘inTransform’ at position 2 and 3 are not required.

The response from Service ‘B’ flows back into the route, and the response goes through Processor ‘C’ where a camel-based transformation (e.g. xquery, XSLT, java code) can be used to transform/enrich the response in any way required. This transformed message can now be returned back to client. The ‘outTransform’ (position 4) is not necessary in this solution.

Using CXF in Generic Provider Mode

One aspect of the CXF transformation feature that is problematic is the requirement that you are restricted to performing transformations that convert message into something that conforms with the WSDL the endpoint is tied to. If the message is transformed in a way that makes it non-conformant, an error is thrown when it reaches the endpoint.

In order to make the use of CXF transform easier within Camel, there is a way to disable the requirement that the incoming message match the operation/binding of the associated CXFEndpoint. That way, if this message-level validation is disabled on the Endpoint you can freely apply tranformations to any format at the endpoint and it will continue to pass through the endpoint without failure.

You should be careful when over-riding this as the purpose of a WSDL is to do the operation checking and some special handling based on the determined operation and also to provide a way for the consumer to get the typed interface information (i.e., retrieving the WSDL from the endpoint). By disabling, you will do neither of these, so you could just simply set up your endpoint using the generic provider mode (i.e., omitting the wsdlURL and serviceClass properties of the endpoint).

In this case, the message will be transformed using CXF's transform feature and directly transferred to your camel route where you can do further modification necessary and set the operationName header to match the operation at the outbound endpoint

In this way, you can allow messages to be transformed using CXF transform feature without the problem of adhering to a particular interface.

Summary

The transformation capabilities of the CXF transform mapping language are limited. The transform language works well for minimal changes such as element name changes or namespace changes. More involved e.g. structural changes to complex types or multi-faceted elements are not easily implemented using the mapping language.

A CXF transform feature is applied to a specific CXFEndpoint in a route. Because the transform is tightly tied to the specific CXFEndpoint, it cannot be used to apply a transformation at random spots on a message flow through the route. This limits the usage of the CXF transform feature as a general mapping tool for transformations within a camel route.

In order to make the CXF transform feature a useful technique for performing general transformations to messages as they flow through a route, you will likely need to disable the opearation validation of messages at the CXFEndpoint and use the generic provider mode. This will allow for further modifications in the transform feature than would normally be possible if you used the default strictly-typed message checks that typically are part of the message entry into the route.

Configuration model for “Environment Aware” Routes 

This document will describe a configuration mechanism for Fuse routes that will simplify deployment. No longer will you need to prompt installer for environment, and update properties in the install script. Instead, the configuration file (.cfg) contains property values  for every environment. The route only reads the properties associated with the environment in which the route runs. This configuration model brings with it the following unique characteristics: 

• Properties file contains values of ALL properties for ALL environments.
• Deployment no longer needs to involve logic to set environment-specific property values.

By limiting the logic and user interaction of the deployment script, there is less chance of making an error in the script. Also, there is a reduction in (scripting) code that must be maintained. Environment specific values are no longer hard-coded into the script, rather these properties have been set up-front in static configuration (.cfg) file(s) that gets deployed with the route(s).

Version Info

This document was written based on JBoss Fuse 6.x and applies to bundles running inside this version of the JBoss Fuse container.

Sample Property file

Let’s take a look at how a property file (.cfg) will look using this model. Notice the file contains properties for every environment (DEV, VAL, INT, PRD, etc.). Each property contains a special identifying prefix to distinguish it as belonging to a specific environment.

The above properties are written in such a way that they are “environment aware” and only the property for the environment Fuse is running in is picked up. The rest of the properties are ignored. Having all properties for all environments may seem excessive, however it could be argued that having property values for every environment available in the configuration file as a reference is useful.

How does Fuse know what environment it is running in?

Fuse will need to be configured to “know” what environment it is running in.  You will need to do a one-time manual edit of a specific Fuse configuration file.

Inside the Fuse configuration file ‘$FUSE_INSTALL\etc\system.propeties’ you can add the following entries.

The two added entries are ‘karaf.environment’ and ‘karaf.instance’ (lines 37 and 38.) Of course, the values of these new system properties should be appropriate for your environment and instance. You may only have a single machine per-environment, in that case, ‘karaf.environment’ is the only system property that needs to be defined.  Add ‘karaf.instance’ if a environments have 2 or more machines, and property values may differ depending on the particular machine (instance) within a given environment. You will need to restart Fuse in order for these environment indicator properties to be available to your bundle’s configuration.

Once Fuse is restarted with the above system properties set, the following entries added to your bundle’s configuration file will now have special meaning (see lines 3 and 4 below).

The configuration file specifies the values of properties by using nested system properties to identify the environment. For example line 3 above the value of ‘to.uri’ is defined by using a nested system property ‘karaf.environment.’

to.uri=${${karaf.environment}.to.uri}

                                  

                  ${DEV.to.uri}

 The ‘${karaf.environment}’ system property (defined in the Fuse ‘system.properties’ file) will resolve at runtime to the environment name e.g. ‘DEV’, and then that is used to designate the prefix for the property i.e. ‘${DEV.to.uri}’. This then resolves to the correct configuration property found further down in the configuration file (see line 7 in snippet from configuration file.) In this example, the true value of ‘to.uri’ becomes the value of the ‘DEV’-prefixed version of this property.

Properties for all environments where the route will be installed will be included in the configuration file. The correct prefix for each environment will match 'karaf.environment' used for the environment and should be specified exactly as it appears in the ‘system.properties’ file of the container in which it appears.) For example: DEV, VAL, INT, PROD, etc.

Using the properties in the Camel Context (Spring XML configuration)

Because the use of the nested properties is confined to the configuration file only, there is nothing special that needs to be done in the Spring configuration for your route in order to use this configuration model.

The spring configuration will refer to the normal (non-prefixed) version of the properties. The environment-specific resolution is confined to the property file,  and does not need to be a concern within the Spring XML of your route.

See lines 27-31 where property placeholders are used as values of properties injected into a Spring bean. See line 37 where the value of a property placeholder is used within the camel route. (Notice the need for double brackets ‘{{‘ and ‘}}’ to de-reference such properties from within the camel route. )

In summary, the use of env-specific properties within your configuration will not affect your route’s code.