Integrate back-end systems through MQ within an ITX Map

The point of this post is to show an example on enhancing data in a file with data from a back-end system. An example could be that a file is received consisting of a Container ID. Based on that Container ID, it should be done a request to a back-end system consisting of a link between the Container ID and the Shipment where that Container is put.

I will use IBM Transformation Extender to do the transformation. WebSphere MQ will be used to communicate with the back-end system. In this example the back-end system is not really a system, but just a Business Process in Sterling B2B Integrator that takes a message from MQ, adds 1.000.000 to the number received and returns it to MQ again. This is just to simulate that a Container ID is received to the back end system and a Shipment ID is returned back.

Test Documents

I start by creating a very simple input file. This file is just made up, and has no root in the real life. It’s just to avoid having to create a more complex map than necessary. The file is a CSV file consisting of two fields. First it is the Container ID, then it is the Container length in feet (I don’t use the last field to anything). It looks like below:

1025;40

The Output file after transformation and lookup in the back-end system should be like below:

1025;40;1001025

MQ Setup

I start with setting up what’s needed in MQ. I need one Queue Manager and two queues. One Queue is for the request to the back-end system from ITX, and the other is for the response from the back-end system to ITX.

Below I have the Queue Manager SFGManager (this is a Queue Manager I use for Sterling File Gateway as well, thus the name). I also have created two Queues for this use case. That is the ITXQueue and the ITXReplyQueue.

MQ Setup

The idea is that ITX will put the Container ID on the ITX Queue. Then the back-end system listens to that Queue, and replies back to the ITXReplyQueue with the corresponding Shipment ID. Then ITX listens to that Queue, and ads it to the output document from the map.

ITX Setup

IBM Transformation Extender is the key component of this example. ITX has MQ Adapters that enables it to communicate directly with MQ. So it is possible to PUT and GET data to/from MQ Queues. This is a nice way of communicating with back-end systems.

Type Trees

I start by creating the Type Trees I need. I need 4 type trees in total. One for the input file, one for the output file. And then I need two additional type trees for the Put and Get to/from MQ.

Map

The Map I create is a map that maps the whole record in the input file to the output file. In my test file I only have one record, so it could probably have been done simpler, but it is habbit. Within the map, I have a Functional Map that does the work for each record in the file.

Below is the Map for the file itself.

Map for the File

Then the Functional Map below, for each record, is where the logic is. It has two input cards and two output cards.

Card In1 (top left) is the input from the record in the file.
Card ToMQ (top right) is the Container ID sent to MQ.
Card FromMQ (bottom left) is the returned Shipment ID from MQ.
Card Out (bottom right) is the combination of the In1 and the FromMQ cards.

Functional Map for each record.

The two cards communicating with MQ has some MQ Adapter logic in the card.

Below is the Properties for the ToMQ Output Card. The TargetRule has the target of “IBM WebSphere MQ (server)” and the command “-QMN SFGManager -QN ITXQueue”. So the Target is an MQ Server, and the Command tells the Output Card to use Queue Manager “SFGManager” and put the result on the Queue “ITXQueue”. In my setup the MQ server is on the same physical server as the ITX. Thus not needing any Server address.

ToMQ Output Card

Below is the properties for the FromMQ Input Card. It has the GET Source set to “IBM WebSphereMQ (server). It has the Command “-QMN SFGManager -QN ITXReplyQueue -LSN {10}”. So the Source for the Input Card is an MQ Server, The Queue Manager is “SFGManager” and the Queue is “ITXReplyQueue”. The “-LSN{10}” command tells the map to listen for a message and do that for either 10 seconds, or until it receives a Message from MQ, whatever comes first.FromMQ Input Card

Back-end system

As mentioned my Back-end system in this example is simulated by a simple Business Process in Sterling B2B Integrator. It has a WebSphereMQ Suite Async Receive Service listening to the ITXQueue (I have a previous post on how to have Sterling B2B Integrator listen to a MQ Queue here). When it receives a message, it runs the message through a map. The map is just taking the number (the Container ID), and adding 1.000.000 to that number. So if the Container ID is 1025, the Shipment ID will be 1.001.025. Then the Business Process is putting the result to the ITXReplyQueue (read this post to see an example on putting a message from Sterling B2B Integrator to MQ). So this is a simple way of simulating a back-end system doing something without actually having a real back-end system.

Testing

The testing of this is basically done by running the map in ITX directly from ITX with the input document as input. It is then possible to see in MQ that a message is put onto the ITXQueue for a very short time before it disappears again. Then it is possible to see in Sterling B2B Integrator that a message is handled in the “back-end” system. Then a message shows up on the ITXReplyQueue, and is taken by the ITX Map, and a Output file containing the Shipment ID is put on the disk.

Things to improve

There is one obvious thing to improve on this example. I am not sending any Correlation ID or similar with the message on MQ. So here I get the correct result back only because I am running one message at a time. If there are multiple messages on the queues, it is not certain the resulting Shipment ID would be the one corresponding to the Container ID sent. To achieve that it is possible to use -CID “Correlation ID” in the Command string used for the Output Card and Input Card used communicating with MQ. But I didn’t find a way of fetching the Correlation ID in my “back-end” system, thus not being able to send it back with the response. But that is probably solved using a real back-end system and not a simulated one.

 

1 comment on “Integrate back-end systems through MQ within an ITX MapAdd yours →

Leave a Reply to Mehari Afewerchi Cancel reply

Your email address will not be published. Required fields are marked *

%d bloggers like this: