Cost Saving with OSGI in Business-to-Business (B2B) Model

Foreword:

One of the projects that I am involved provided Business-To-Business Service Model.

The main idea is to provide the services that are normally only available for internal processes of the company, to the external customers of the company.

It is basically providing an external interface build via web service to access company’s internal ESB.

This is a very standard scenario that is discussed in many articles; it is a well known business practice. What I want to point out here how this model suffers from some basic business requirements and how these requirements can be fulfilled in a more cost effective way.

If you observe several B2B systems, you will see that a common pattern immerges. You are providing your services to external company that has a different development cycle then yours, direct consequences of this, if you make changes to your services because of internal requirements, external partners may not have the budget or necessity to implement them (for ex you change an existing process which now needs an additional address information but the external partner will never need those changes because he doesn’t use those).

If only one version of your services exists, your external partners are forced to implement those changes and this cause costs, because of this necessity several B2B projects provides several versions of those services for backward compatibility and a service level agreement for the time to live of those services, SLA will say that there will be no more then 3 backward compatible versions of it.

When conditions are met the older versions will be phased out.

Now you might read until this point and say what is the big deal about it? Well the project that I was in, was big SOA project that had several Partner Systems with their internal release cycles so when we want to deliver the actual versions of B2B interfaces, we would have to make up to 10 deliveries to the production systems per year.

Well number the deliveries were naturally a problem but the actual biggest cost was the testing of these interfaces. If we would have 20 partner systems and 10 delivery dates those 20 Systems has to be all tested for these 10 deliveries and this was a huge cost, specially considering may be out of those 20 system only 2 has real changes during one delivery.

Substantial cost savings were possible if we could test several delivery packages at one date and deliver on there dedicated delivery dates.

Unfortunately conventional middleware platforms this is not that easy. We could only deliver one version of the application to the production (because middleware software, application servers only able to hold one version of the binaries) until OSGI technology appeared.

OSGI technology give us the possibility of deploying several version of an application simultaneously, they can be all active at once or there can be activated/deactivated on demand.

So what OSGI bring us to solve our problem.

OSGI is a Service Platform implementing a dynamic component model which normally does not exist in Java Virtual Machines. In OSGI we can define our Java Libraries as Bundles and deploy them. Bundles are heavily versioned with descriptors, it might contain the same Java Classes but it is interpreted by OSGI runtime as different Java Libraries.

So what this technology presented us an opportunity.

As you remember our problem is that we have high cost while we have 10 different delivery dates per year so we have carry the cost builds, testing and deployment of our application 10 times per year.

What if we know the version 1,2 and 3,4 and 5,6 and 7,8 and 9,10 of our application, will change so little in between them, we can actually test them together and deploy them together but activate them in different dates. OSGI will exactly give this possibility to do this.

I can tell you testing and deployment of an enterprise quality application is expensive undertaking, cutting cost of 10 test/deployment cycle will be a considerable cost saving.

Problem and Solutions:

So what will be the problems that we will encounter and how can we solve them.

First of most of the B2B Services presented as Web Services and Web Services are strongly controlled with their WSDL. That means if we are going to change the version of a B2B Service, most probably we will also change its WSDL and inevitably configuration changes at Service Customers. Most of the Service Consumers don’t like this because many of them have quarterly releases and if our application has 10 releases why they should increase the number of releases and their costs, just to stay compatible with our application.

Secondly most of the B2B Service keeps 2 or more versions of them up to provide backward compatibility so a new release from us should not override or backward compatible services.

So we have 2 major requirements, for once we need an OSGI platform dependency and for the second we should be able to present our B2B Service to our customer’s version independent.

There is one platform that can present these two demands, it is Fuse ESB.

Fuse ESB is the commercial version of the Apache Servicemix Open Source project. You can use it for free and buy commercial support when you need. So you might ask why we need an ESB.

Now what are we going to do is to define an entry point in a Fuse workflow and also configure the workflow to redirect call to our B2B Service to the specific versions.

Below is the picture which is showing how the most B2B Services deployed currently.

1

Service Consumer has 3 URLs to the versions of the Service. From the Service Consumer point of view if a new service introduced or an old one removed, Service Consumer should reconfigure it systems, even that is not in its development plans.

Now in this second picture we see that Service Consumer is speaking with a central point for the communication, if a new service is introduced or removed it would be totally transparent for the Service Consumer.

2

So how are we going to achieve this? As I give hints above, we are going to use Fuse/Servicemix as an ESB which will intercept call to our B2B Services and redirect them to desired version with the information included in the calls. While Fuse/Servicemix based on OSGI we can turn on/off versions of our Services while staying completely transparent to the Service Consumer.

Feasibility Study:

I prepared a feasibility study project with the above mentioned concepts.

Following 2 screenshots (screenshots are from Karaf, management UI for Fuse) will show you that OSGI is the right choice here.

3

4

When you are looking to the previous pictures you will see three interesting details. When you check feature list you see that ‘osgi-b2b-router-feature’ has field written ‘installed’ which can configured on runtime to ‘uninstalled’ which will remove the feature.

Also if you look to the OSGI Bundles list you will see that near to the ‘OSGI B2B Router’, ‘OSGI B2B Mediation’, ‘OSGI B2B Producer’ Active field, so you can turn bundles on/off (osgi:start/stop command) on the runtime.

This is principle part of our idea/theory so we can install different routing versions for us or different versions of our b2b services and turned on/off those as it us pleases.

To be able to demonstrate what we are talking about here functions, I built a SOAP UI project which is simulating our B2B Service and the consumer for it.

5

Above screenshot displays the Mockservice that we created for simulating B2B Service.

6

And this one simulates the customer, as you might see we called the Version 1 of our service and received the answer.

Now lets call the same URL with the version 2 of our service, we have our Version 2 of our services simulated on port 9090.

7

You can see that mediation layer automatically detects that this is the new version of our service and directs the request correctly to the version 2 of the service.

We modified the service so that now it delivers a date field in the response, if you want to be sure that it is not exactly the same answer.

Now you can say, what is so special about it every Enterprise Service Bus can do this, then consider the following screenshot.

8

The screenshot belong to ‘Karaf’ OSGI management console of Fuse, you see there 2 version of our services OSGI B2B Producer 1.0.0.SNAPSHOT and 2.0.0.SNAPSHOT. The Active status of those bundles indicate both are deployed and active in the same time. We can turn those bundles on/off on the runtime or even install new version of those bundles without end user discerning anything.

That is the power of OSGi and the solution we propose here.

Project Structure:

This is how the project structure looks like.

9

A short summary of the project functionalities will look like the following.

osgi-b2b-router this is the central entry point for our B2B Services, this component will receive the initial request from Service Consumer and redirect to the mediation so it can be forwarded to the correct version.

osgi-b2b-mediation this project will inspect the incoming message and will decide which version of the B2B service that we provide should serve this request.

osgi-b2b-producer will call the real implementation behind our service and deliver the response (name producer is a concept in Servicemix and its Webservice implementation stack which I will go in detail later).

osgi-b2b-feature is a construct of the Fuse framework and is actually a deployment unit which I will also explain it later stages.

Detailed Analysis of Projects:

osgi-b2b-router:

This is how we configured our entry point to our project.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:osgiRouter="http://osgi.salgar.org/osgirouter"
	xmlns:orderProvider_v1="http://v1.salgar.org/B2BService"
	xmlns:http="http://servicemix.apache.org/http/1.0"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
						http://servicemix.apache.org/http/1.0 http://servicemix.apache.org/http/1.0/servicemix-http.xsd">

	<http:consumer service="http:FindOrders" endpoint="httpEndpoint" locationURI="http://localhost:8088/MockB2BService"
		defaultMep="http://www.w3.org/2004/08/wsdl/in-out" targetService=" osgiRouter:mediation"  />

	<bean class="org.apache.servicemix.common.osgi.EndpointExporter" />
</beans>

You can see that we configured our central entry point with a http component of Servicemix(http:consumer).

This is the component when configured that will receive the message arriving to our URI and port defined in the locationURI property. In thise case we are listening for the localhost port 8088.

Later it will route this message to the osgiRouter:mediation target enpoint which will then decide to which version of services has to be called.

osgi-b2b-mediation:
This project will receive the message from the router project and inspect it. It will specially analyze the namespace property of the web service call which will indicate the intended version of the Service Consumer.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns:eip="http://servicemix.apache.org/eip/1.0"
	xmlns:osgiRouter="http://osgi.salgar.org/osgirouter"
	xmlns:orderProvider_v1="http://v1.salgar.org/B2BService"
	xmlns:orderProvider_v2="http://v2.salgar.org/B2BService"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
						http://servicemix.apache.org/eip/1.0 http://servicemix.apache.org/eip/1.0/servicemix-eip.xsd">

	<eip:content-based-router</strong> endpoint="osgiRouterMediation"
		service="osgiRouter:mediation">
		<eip:rules>
			<eip:routing-rule>
				<eip:predicate>
					<eip:xpath-predicate id="wsdl_version_predicate_v1" 				xpath="(//namespace::*[.='http://v1.salgar.org/B2BService']) = 'http://v1.salgar.org/B2BService'" />
				</eip:predicate>
				<eip:target>
					<eip:exchange-target service="orderProvider_v1:findOrders" />
				</eip:target>
			</eip:routing-rule>
			<eip:routing-rule>
				<eip:predicate>
					<eip:xpath-predicate id="wsdl_version_predicate_v2"				xpath="(//namespace::*[.='http://v3.salgar.org/B2BService']) = 'http://v3.salgar.org/B2BService'" />
				</eip:predicate>
				<eip:target>
					<eip:exchange-target service="orderProvider_v3:findOrders" />
				</eip:target>
			</eip:routing-rule>
		</eip:rules>
	</eip:content-based-router>

	<bean class="org.apache.servicemix.common.osgi.EndpointExporter" />
</beans>

At this endpoint an eip:content-based-router component evaluate the rules defined for this component and an eip:predicate component deciding based on the xPath statement (analyzing the names spaces of calling Webservice to decide which version of our Service is called, ‘orderProvider_v1:findOrders’ version  or the ‘orderProvider_v2:findOrders’ version).

osgi-b2b-producer:
Producer is a notation used on the CXF Web Service framework. This will be the component responsible to call the the version of the service we provide, in Servicemix .

It’s configuration would look like the following.

<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:cxfbc="http://servicemix.apache.org/cxfbc/1.0"
	xmlns:orderProvider_v1="http://v1.salgar.org/B2BService"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
						http://servicemix.apache.org/cxfbc/1.0 http://servicemix.apache.org/cxfbc/1.0/servicemix-cxf-bc.xsd">

	<cxfbc:provider endpoint="findOrdersProviderEndpoint"
		useJBIWrapper="false" wsdl="classpath:v1/B2BService.wsdl" service="orderProvider_v1:findOrders"
		locationURI="http://localhost:8989/MockB2BService" />	

	<bean class="org.apache.servicemix.common.osgi.EndpointExporter" />
</beans>

We define the wsdl of the Web Service this producer configured for (classpath:v1/B2BService.wsdl) and URL that the Web Service responds(http://localhost:8989/MockB2BService).

In a real life project, 2nd version would be created in another Git branch and deployed from there but if I create a branch for this feasibility project, it will complicate the things too much for you, if you try download and install this project. So I just created another project under the name osgi-b2b-producer-v2 and placed under Git so it can simulate the 2nd version of our project.

osgi-b2b-feature:
This is a Servicemix proprietary artefact, a deployment feature from Servicemix. We can deploy Servicemix independent bundles or we can deploy several bundles configured to form a feature.

For this we need special file (osgi-router\osgi-b2b-feature\src\main\resources\ osgi-b2b-router-features.xml) configuring the feature looks like following.

<features>
	<feature name="osgi-b2b-router-feature" version="${osgi-router.version}">
		<feature version="${servicemix.version}">servicemix-cxf-bc</feature>
		<feature version="${servicemix.version}">servicemix-http</feature>
		<feature version="${servicemix.version}">servicemix-eip</feature>
		<bundle>mvn:org.salgar/osgi-b2b-router/${pom.version}</bundle>
		<bundle>mvn:org.salgar/osgi-b2b-mediation/${pom.version}</bundle>
		<bundle>mvn:org.salgar/osgi-b2b-producer/${pom.version}</bundle>
	</feature>
</features>

Please pay special attention to the notation used for the bundles mvn:org.salgar/osgi-b2b-router/${pom.version}. Servicemix is closely integrated with Maven and with previous notation it can find artefacts from Maven repositories and install it.

The feature artefact is also really usefull notifiy Fuse if our Bundles are dependant of other artefacts or not, as you can see for ex. with ‘servicemix-http’. When Servicemix notify this it will locate and activate the Servicemix Http Component for us.

There is one additional thing that we have to do to help our Maven build to correctly build the feature artefact, following elements has to be defined in the Maven pom.

<plugin>
	<groupId>org.codehaus.mojo</groupId>
	<artifactId>build-helper-maven-plugin</artifactId>
	<executions>
		<execution>
			<id>attach-artifacts</id>
			<phase>package</phase>
			<goals>
				<goal>attach-artifact</goal>
			</goals>
			<configuration>
				<artifacts>
					<artifact>
						<file>target/classes/osgi-b2b-router-features.xml</file>
						<type>xml</type>
						<classifier>features</classifier>
					</artifact>
				</artifacts>
			</configuration>
		</execution>
	</executions>
</plugin>

This configuration will create a special artefact ‘osgi-b2b-router-features.xml’ which we will need when we like to install the feature to the Servicemix.

Deployment:

Servicemix is closely integrated with Maven, it can deploy features and bundle from Maven Repositories with the following command.

features:addurl mvn:org.salgar/osgi-b2b-feature/1.0-SNAPSHOT/xml/features

You see below is a screen shot of Karaf, management console for Servicemix OSGI platform. You can see here with the features:list command we see the all available features. Our one is also visible there, osgi-b2b-router in this case is already installed, otherwise we have to use features:install command to install it.

10

Below you can see the installed the bundles from it, the moment when we install the features all the bundles that are included are also installed.

11

Fuse/Servicemix Configuration:

We have to make some home work to configure Fuse so it can work with Maven Repositories seamlessly.

We have to configure the file org.ops4j.pax.url.mvn.cfg which lies in ‘fuse\etc’ directory.

You have to modify the following property.

org.ops4j.pax.url.mvn.localRepository

You should have set the property to your local maven repository.

One point, I also have to mention here, there are lots of pre-installed features/bundles in the Fuse but to keep resource usage low, they are not activated until they are needed. So when we install our feature it will try to access to those features and while we did configure necessary dependencies in our feature artefact Fuse will start-up without any problem.

<feature version="${servicemix.version}">servicemix-cxf-bc</feature>
<feature version="${servicemix.version}">servicemix-http</feature>
<feature version="${servicemix.version}">servicemix-eip</feature>

If you need any additional features you should I either start those manually or also place in feature artefact.

Conclusion:

I hope I was able to show you that implementing an interesting technology and Enterprise Integration Pattern can really help saving lots of cost. With above mentioned methods you can test several versions of your application in a moment in time and deploy on the production and turned on/off those on runtime with zero downtime or even make bug fixes with zero downtime.

Appendix:

Preparing Project Environment:

Getting Fuse:
You can download the Fuse from the following URL:

http://www.jboss.org/download-manager/file/jboss-fuse-6.1.0.GA-full_zip.zip

After you unzip the file and make the maven configuration that I mentioned before.

You can start the fuse with the following command ‘fuse\bin\fuse.bat’.

Getting Project:
The project place under the GitHub, you can download it from it with any GitClient.

I personally use cygwin (windows unix emulation) and do it with command line instructions. If you are windows operation system user and more comfortable with user interfaces you can use the following tool (http://code.google.com/p/tortoisegit/)

If you use cygwin or any git shell version you can get the project with the following command:

git clone git@github.com:mehmetsalgar/osgi_router.git

Maven Setup:
The build system we used for the project is Maven which you can download from the following URL (http://maven.apache.org/download.cgi)

If you are not an experienced with Maven, one point you have to be careful if you are working behind company firewall, you have to configure the following lines in setting.xml in conf directory of your Maven installation.

<proxies>
<!-- proxy
| Specification for one proxy, to be used in connecting to the network.
| -->
<proxy>
<id>optional</id>
<active>true</active>
<protocol>http</protocol>
<username>xxxxx</username>
<password>xxxxxx</password>
<host>some.proxy.com</host>
<port>3128</port>
<nonProxyHosts>127.0.0.1</nonProxyHosts>
</proxy>
</proxies>

If you are working with windows, it might be wise to change your maven repository location from your MyDocuments to a simpler path because some classpaths becomes too deep and have problems with 255 character limit of windows. So it is better to select a shallow directory position for your repository, for ex . c:/repo

<localRepository>c:/repo</localRepository>

Now if you execute the following command from the directory where your root pom is lies, you should see the following output which means you successfully built the project.

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] OSGI Router ....................................... SUCCESS [0.250s]
[INFO] OSGI B2B Router ................................... SUCCESS [2.391s]
[INFO] OSGI B2B Mediation ................................ SUCCESS [1.859s]
[INFO] OSGI B2B Producer ................................. SUCCESS [2.000s]
[INFO] OSGI B2B Producer ................................. SUCCESS [1.203s]
[INFO] OSGI B2B Feature .................................. SUCCESS [1.641s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10.578s
[INFO] Finished at: Tue Aug 26 19:17 CEST 2014
[INFO] Final Memory: 26M/495M
[INFO] ------------------------------------------------------------------------

SOAP UI:

SOAP UI is a really useful tool to test web service interfaces.

When you have the wsdl’s of the web services you want to test, you can start a SOAP UI project and calls those this web services with the sample requests.

You can download the SOAP UI from the following URL (http://dl.eviware.com/5.1.2/SoapUI-Pro-5.1.2-win32-standalone-bin.zip?_ga=1.124456330.1225656795.1409838533)

I also created a SOAP UI Project to test our application, if you downloaded the project from github, you can find the project in the following directory ‘osgi-router/osgi-router-soapui-project.xml’.

Servicemix

ServiceMix is an enterprise-class open-source distributed enterprise service bus (ESB) based on the service-oriented architecture (SOA) model.

Servicemix use Spring for it configuration purposes. If you want to configure components and endpoints for your project, you should create a Spring xml configuration files calles beans.xml and place under the following directory in your project ‘src\main\resources\META-INF\spring’, then it will automatically identified and initialized by Servicemix.

It has several components that you can activate by only including their namespaces to the beans.xml file, like the following.

“xmlns:eip=”http://servicemix.apache.org/eip/1.0&#8243;,”xmlns:http=”http://servicemix.apache.org/http/1.0″”, “xmlns:cxfbc=”http://servicemix.apache.org/cxfbc/1.0″”.

This will inform Servicemix to load and initialize this component (Off course this component must be in installed state in your OSGi container) .

You can see the list of available Servicemix components in the follwing URL Servicemix.

Basic working principle of Servicemix components that you define Endpoints with them, when a Servicemix component finishes it work it send its result message to the endpoint defined in the targetService property. It follow this chain to the end and then send the result to the consumer.

Servicemix documentation will indicate namespaces for the components.

OSGI

OSGi (Open Service Gateway Initiative) defines a service plaftorm for Java which is modular system allowing Java to behave like complete and dynamic component model. That is something that does not exist in standart Java/VM Environments.

It makes possible that several version of Java Package/Classes exist in parallel/modular fashion. These Java Classes/Package are deployed in the form bundles which can be installed, started, stopped, updated and uninstalled without requiring a reboot.

Service registries also allows bundles to detect addition or removal of the services on the runtime.

Building blocks of OSGI consist of Bundles, which are group of Java classes and additional resources equipped with a manifest (MANIFEST.MF) which are describing their functionality and dependencies, Lifecycles defining state of a Bundle (Installed, Resolved, Starting, Active, Stopping, Uninstalled) and Services which can be registered under any Java Interface.

OSGI Bundles and Maven

Although the way how Maven and OSGI handle the bundles are quite similar there are still differences.

In a Maven pom, a dependency definition looks like following.

<dependency>
	<groupId>org.apache.servicemix</groupId>
	<artifactId>servicemix-common</artifactId>
	<version>${servicemix.version}</version>
	<scope>provided</scope>
</dependency>

We define here which groupId and version this artefact belongs.

Things are little bit different for OSGI, the dependencies are defined a file called MANIFEST.MF as looks like as following (I will left out some details because these files can be huge in size).

Import-Package:
javax.jbi.component;version="1.0",javax.jbi.management;version="1.0",javax.jbi.messaging;version="1.0"

As you see the notation is different but the information it contains is still quite similar. GroupId, artefact and version, unfortunately Maven and OSGI are not totally compatible without the use of some Maven plugins (like Tycho) they can totally solve their dependencies.

But fortunately there are some Maven repositories built with idea of creating Maven and OSGI compatible artefacts, so they contain the pom.xml and MANIFEST.MF information at the same time.

If you look to the our root pom, you will see a configuration element introducing one of those repository.

<repository>
         <id>fuse</id>
	 <name>FUSE Repository</name>
	 <url>http://repo.fusesource.com/maven2/</url>
	 <snapshots>
		<enabled>false</enabled>
	 </snapshots>
	 <releases>
		 <enabled>true</enabled>
	 </releases>
</repository>

If you download this project and want to build it Maven, you should check that you have internet access to this repository.

Advertisements

About Mehmet Salgar

Mehmet Salgar
This entry was posted in B2B, Business to Business, cxf, EIP, Enterprise Integration Pattern, ESB, Fuse, Karaf, Maven, Open Source, OSGI, Servicemix, SOA. Bookmark the permalink.

One Response to Cost Saving with OSGI in Business-to-Business (B2B) Model

  1. Pingback: Micro Services and JBoss Fuse/Apache Servicemix | Mehmet Salgar's Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s