• Java / JEE middleware and open source integration technology (Mule ESB, Talend, ServiceMix (Fuse), Camel)
• .NET, WCF and MS Biztalk integration technology
• project management and agile software development
• strategic business technology consulting

The new employees will further enhance greenbird’s competance and capacity for implementation of innovative smart metering solutions and advanced business integration systems.

making data fly.
by greenbird.com.

In order to create a new map you have to create an EAI-project. If you haven’t downloaded the CTP you may do it from here. A map can be found under the ServiceBus-template.

Create map (Click to enlarge)

Pay attention to the map extension, its .trfm which is an acronym for “transform”. In BizTalk a map has an extension .btm which is short for “BizTalk map”. When a Service Bus map is created the following picture appears to us.

Empty map (Click to enlarge)

By now any BizTalk 2010 developer will recognize the Mapper-design. It looks exactly like the one found in the latest BizTalk release but without tabs. This is a positive surprise for us BizTalkers since this design resembles the well-known and improved version of the BizTalk Mapper but the lack of tabs will make maps less user-friendly when dealing with complex transformations, for instance huge EDI messages.

This is where the “total” similarity ends. If we take a closer look at the toolbox we will see that the icons for functoids are different, some functoids are new and some even got new names.

SB-toolbox(left) vs. BT-toolbox (Click to enlarge)

To make it straight, a functoid in a Service Bus Mapper is called an operation. Operations have similar functionality as functoids in BizTalk and are easily removed or added in the toolbox, just as the functoids. The one operation I tried to find when the toolbox opened for the first time is the one resembling the Scripting-functoid, from which you can call methods stored in external dlls or type in C#/VB/JS/XSL expressions. Unfortunately, this one is absent in this release but I’m convinced that it will show up in the upcoming releases since a more complex mapping is practically impossible without it.
The one particular feature I thought was slick is the way sub-looping operations are presented on the Mapper. “MapEach Loop” and ”ForEach Loop” are two new looping operations presented as “containers” (see pic bellow) and when adding new loops inside each one of them they will be hosted in the parent loop’s container. This is much more intuitive than it is in the BizTalk Mapper because you may visually distinguish between parent/children-loops.

Sub-looping (Click to enlarge)

The most critical difference between a Service Bus map and its BizTalk counterpart is their map formats. If you open both maps in a XML editor you will see that they are completely different, making them totally incompatible with each other. In other words it is pretty much impossible to migrate a BizTalk map to Service Bus map. Since they are both stored as XML maybe we could create a one-to-one mapping in BizTalk in order to convert them (a map to convert a map to another map):D

Service Bus map(left) vs BizTalk map (Click to enlarge)

As in BizTalk project, we can right-click on a map and choose “Test Map” in the context menu. However, no XSL-file path is displayed in the Output-window and I could not find any XSL-file in the temp folder. I was also unable to locate any article on the Internet which confirms that XSL is used as the underlying mapping technology. In addition, there is no way to specify a custom XSL-file which is pretty lousy. Why is it lousy? I prefer custom XSL over Designer and functoids anytime I work with complex transformations but I have to admit that I didn’t give BizTalk 2010 Mapper much chance.
The other thing I could not find is the way operations are created but I imagine it should be similar to functoids but with new class(es).

To summarize my first impression of the Service Bus Mapper CTP, there are three positive and three negative characteristics.

Positive:

• Intuitive sub-looping
• Mapper resembles BizTalk 2010 Mapper (design, “Test Map”-functionality)
• Nice looking operation icons

Negative:

• Lacking “Scripting”-operation
• Tab-less Map designer
• Cannot use custom XSL mapping (unknown underlying transformation technology)

I realize Microsoft promised to deliver Azure AppFabric integration tools in 2011 and they did, just before the year ended. It was maybe a bit rushed but the Mapper looks promising even though it is missing some cruitial features. However, this is not production ready release and there is still a hope they will add those missing features later on.

Thanks to everybody for a great year with

• innovative smart integration solutions.
• innovative smart metering systems and smart grid applications.
• challenging architecture and integration projects.

With the best wishes for 2012!

making data fly.
greenbird.com.

Introduction
greenbird are working with Europe’s leading Credit Management Services company, Intrum Justitia, to create a SOA enabled integration solution targeting the energy industry in Norway.
The solution expose a set of generic APIs plus a number of client specific adapters and integrates with a back end debt collection system by exchanging flat files. A large amount of integration points and data transformations are involved.

The solution is implemented on webMethods Integration Server from Software AG, a best of breed integration platform with a “point and click” based IDE called webMethods Developer.

Integration Server has great support for SOA enablement and application integration. It also has great support for manual testing, including a debugger with stepping and breakpoint capabilities.
It does however not come with built in support for automatic testing.

We are following an agile process and must be able to continuously handle new and changing requirements. Due to the “point and click” nature of the Integration Server IDE a lot of manual work is involved when refactoring existing features and adding new ones. It was therefore even more important than usual for us to write robust unit and integration tests to discover bugs and regressions as early as possible.

We wanted to use a testing framework to make the process of writing and maintaining high quality tests as effortless as possible. There are a few commercial frameworks that enables the writing of automatic tests directly in Integration Server but we wanted to leverage existing infrastructure and skill-sets based on Java and the JUnit framework. We found an Open Source JUnit based framework called wmUnit but the project has been inactive since 2006 and lacked some of the features we wanted. In the end we decided to write a simple framework ourselves.

These were the main requirements for our framework:

1. The tests must be able to run in a Continuous Integration set up.
2. We must be able to write the tests as regular JUnit tests.
3. The plumbing involved when doing remote calls to the Integration Server must be invisible.
4. It must be very easy to produce input data.
5. It must be very easy to perform assertions on the data returned by the Integration Server.

The requirement regarding Continuous Integration support was easily attained by fulfilling the requirement that the tests should be regular JUnit tests. There is a whole plethora of CI products supporting JUnit based projects.

The rest of this article gives an overview of how we implemented the other requirements.

JUnit based tests

We created a generic JUnit based superclass that makes the utilities described below easily available for test subclasses. We based the framework on the latest version of JUnit to ensure that the tests could be written using modern paradigms such as test annotations and Hamcrest matchers.

Transparent log-in and session management

The superclass takes care of logging into the right Integration Server and maintaining a logged on Context for the tests.

Transparent invocation and service selection

By using a convention over configuration approach the superclass is able to determine what Integration Server service is under test simply by analyzing the fully qualified name of the test class.
The convention is that the tests are placed in a package reflecting the folder structure of the service and named according to the following pattern: <service name>Test.
E.g.: To test the service my.folder.structure:myService you create a test with the following fully qualified name: my.folder.structure.MyServiceTest.
This enables tests to invoke the service under test with no configuration what so ever:

invoke(inData);

Cursor management

Integration Server returns data in IData structures. IData requires that you access data by opening a cursor. The cursor needs to be destroyed after use. The test superclass manages the top level cursor so that simple return values can be looked up without thinking about resource management:

IDataCursor responseCursor = invoke(inData);
String result = getString(responseCursor, "result");

IData builder DSL

The IData programming model is very verbose and requires resource management. Building even the simplest multi-level input documents is time consuming and error prone.
As an example we’re going to build an IData structure reflecting the following document model:

<rootElement>
        <subElement>string value</subElement>
        <subDocument>
                <name1>value 1</name1>
                <name2>value 2</name2>
         </subDocument>
</rootElement>

This would require the following IData code:

IDataCursor cursor = null;

IData subDocument = IDataFactory.create();
cursor = subDocument.getCursor();
IDataUtil.put(cursor, "name1", "value 1");
IDataUtil.put(cursor, "name2", "value 2");
cursor.destroy();

IData rootElement = IDataFactory.create();
cursor = rootElement.getCursor();
IDataUtil.put(cursor, "subElement", "string value");
IDataUtil.put(cursor, "subDocument", subDocument);
cursor.destroy();

IData document = IDataFactory.create();
cursor = document.getCursor();
IDataUtil.put(cursor, "rootElement", rootElement);
cursor.destroy();

We wanted to be able to easily define smaller documents directly in the test code so we created a simple Java based Domain Specific Language (DSL) that hides the complexities of the IData API. Here’s our example document model implemented using the DSL:

IData document = doc(
        el("rootElement", doc(
                 el("subElement", "string value")
                .el("subDocument", doc(
                         el("name1", "value 1")
                        .el("name2", "value 2")
                ))
        ))
);

The DSL enabled us to quickly define documents, elements and lists without worrying about the IData programming api and resource management.

Converting XML to IData

The DSL is suitable for smaller input documents, but when the document size get bigger it’s usually sensible to maintain the test data separately from the test code. Integration Server provides a couple of services for for converting XML to IData (pub.xml:xmlStringToXMLNode and pub.xml:xmlNodeToDocument). We leveraged these services to enable the tests to store larger input documents in an intuitive XML format. Our example document can simply be saved on the format described above:

<?xml version="1.0" encoding="UTF-8"?>
<rootElement>
        <subElement>string value</subElement>
        <subDocument>
                <name1>value 1</name1>
                <name2>value 2</name2>
         </subDocument>
</rootElement>

The input data structure can then be loaded into the test using the following one-liner:

IData inputData = toIData("my_test.xml");

Converting IData to XML for easy querying

Querying an IData document using the Integration Server Java API is just as verbose and error prone as building one. We decided we rather wanted to work with XML since there are many Java tools for querying XML documents.
Again we leveraged an existing Integration Server service (pub.xml:documentToXMLString) to convert incoming IData structures to XML documents.

We chose the XPath utility from the Java SDK as our query tool. We also created a little wrapper for it (XPathRoot) to make it simpler to use.

This made it easy for tests to query even the most complex IData document using xpath:

String xml = toXML(myData);
XPathRoot docRoot = xPath(xml);
String value2 = docRoot.value("/rootElement/subDocument/name2");

..or if you prefer a one-liner:

String value2 = xPath(toXML(myData)).value("/rootElement/subDocument/name2");

FTP support

It is very easy to set up an FTP endpoint on Integration Server so that services can be called with the content of files received from clients. Integration Server can also return the result of the service invocation as a file with a name on the pattern <input filename>.out. This makes FTP a viable transport for synchronous request/response style service invocations.

We created a simple way to test our synchronous FTP endpoints by integrating the Apache Commons Net FTP client into our test superclass. Files can now be transmitted and the response received by a one-liner:

String resultXml = ftpStore("file_name_to_put.txt", inputStream);

Mule Studio is an Eclipse based IDE where you graphically can create your Mule ESB flows and easily modify them as your integration project comes along. It also integrates nicely with Maven.

Mule Studio supports 2-way editing giving:

• Faster development for experienced Mule users
• Not a black box: Move back and forth between graphical representation and XML
• No need to choose between ease of use and control

For more info, have a look at the presentation or try Mule Studio yourself..

To meet current and future needs for automated business processes and seamless integration, greenbird Integration Technnology and Intrum Justitia have established a new holistic integration platform targeted for the energy industry.

The solution is fitted for Smart Metering / AMI requirements and future Smart Grid possibilities supporting fully automated smart metering integration processes. While the solution is established with a particular focus on the energy industry, it can easily be utilized toward other industries. This gives a more secure, flexible and uniform cooperation between Intrum Justitia and their customers in other industries.

For more info regarding Intrum Justitia’s inkasso solution for energy, please visit greenbird showcase: Intrum Justitia inkasso integration or contact info[a]greenbird.com.

Attending the 2011 European Metering, Billing / CRM conference, greenbird will analyze and summarize the event in a free report focusing on IT related topics such as smart metering security, metering systems, meter data management, event management, smart integration, communication trends or standardization activities.

Register here for your free copy of greenbird’s report from the 2011 European Metering, Billing/CRM conference:
[contact-form-7]

Interested in greenbird’s Smart Metering Integration Platform?
Please visit greenbird Energectis or contact info[a]greenbird.com to arrange a meeting.

For more info, please contact info[a]greenbird.com.

The new offices will also house greenbird’s Smart Integration Lab which hosts a preconfigured demo and testing playground for with both greenbird’s Smart Metering Platform greenbird Energetics and other commonly used integration middleware platforms.

greenbird’s new HQ address – from Aug. 1st: Storgata 8, N-0155 Oslo.

Come around, drop in and visit greenbird or just contact info[a]greenbird.com.

greenbird Integration Technology AS has joined IKT Norge’s “Forum for Smart Energy”. greenbird will contribute with expert knowledge within design, development and implementation of smart metering, smart home and smart grid applications utilizing advanced integration middleware technology.

For more information, please visit IKT Norge’s Forum for Smart Energy or contact info[a]greenbird.com.