Apache Distributed OSGi

OSGi is very famous technology in the standalone applications like eclipse due to its dynamic class loader and interface implementation concepts. OSGi has released enterprise specification and apache provided implementation which CXF-DOSGi. My idea is to create a highly distributed SOA environment and with help of open source technologies and my target architecture is something like below,

Instance 1:  Contains a standalone HelloService implementation
Instance 2:  Enterprise Service Bus setup, with Camel (Enterprise Integration Pattern) configuration.
Instance 3:  Zookeeper Server - Centralized service registry.

Zookeeper Server Setup (Instance 3):

Download zookeeper from apache site and which will be used as centralized service registry for all services. All stored services will be used or accessed by the distributed applications. 

Zookeeper is part of Hadoop project and maintains the services in the cluster of servers. Unzip the zookeeper in any directory and create zoo.cfg file in the [zookeeper-dir]/conf with following configuration.

 
tickTime=2000
initLimit=10
syncLimit=5
dataDir=c:\zookeeper
clientPort=2900

Clientport determines the port in which DOSGi service will use to store the service and lookup the service.  Then need to start the zookeeper server and listen for the client connections, in the bin directory execute following command,

> zkServer

You notice logs for creating new zookeeper snapshots. In this setup we used minimal zookeeper configuration not used distributed server setup.

Hello Service Implementation Setup (Instance 1):

Like to have a Hello Service Interface with “sayHello” method and convert that interface into OSGi bundle and deploy in the Karaf environment. Karaf is OSGi execution environment with excellent shell interface and command framework. You can download the Karaf from apache site, once downloaded unzip in suitable location. 

Set the JAVA_HOME in the environment (if is not set already), with 1.6 version. Karaf not yet certified with 1.7. Run the Karaf command from the bin directory.

>karaf

If the environment is fine, Karaf console welcomes with following JLine text.

E:\SOA\apache-karaf-2.2.6\bin>karaf
  __ __    ____
    / //_/____ __________ _/ __/
   / ,<  / __ `/ ___/ __ `/ /_
  / /| |/ /_/ / /  / /_/ / __/
 /_/ |_|\__,_/_/   \__,_/_/

  Apache Karaf (2.2.6)

Hit '' for a list of available commands
and '[cmd] --help' for help on a specific command.
Hit '' or 'osgi:shutdown' to shutdown Karaf.
karaf@root>

Install the necessary DOSGi bundles to setup the distributed environment. Download apache CXF-DOSGi multi bundles from the apache site and copy the jars to karaf-dir/deploy directory for hot deployment. 

If all dependencies are resolved following bundles should be running in your Karaf environment. 

Zookeeper needs to make connection to server with port 2900 as we configured in the zoo.cfg file. To do so add into karaf-install-dir/etc/system.properties with “org.apache.cxf.dosgi.discovery.zookeeper.port = 2900” configuration.


[ 108] [Active  ] [ ] [    ] [   80] AOP Alliance API (1.0.0)
[ 109] [Active  ] [ ] [    ] [   80] Apache Commons Logging (1.1.1)
[ 110] [Active  ] [ ] [    ] [   80] Apache Log4J (1.2.15)
[ 111] [Active  ] [ ] [    ] [   80] JDOM DOM Processor (1.1.0)
[ 112] [Active  ] [ ] [    ] [   80] SLF4J API (1.5.10)
[ 113] [Resolved ] [ ] [    ] [   80] SLF4J Jakarta Commons Logging Binding (1.5.10)
[ 114] [Active  ] [Created  ] [    ] [   80] Apache CXF Minimal Bundle Jar (2.5.2)
[ 115] [Active  ] [ ] [    ] [   80] CXF Zookeeper-based Discovery Service Bundle (1.3.1)
[ 118] [Active  ] [ ] [    ] [   80] CXF Local Discovery Service Bundle (1.3.1)
[ 119] [Active  ] [ ] [Started] [   80] CXF dOSGi Remote Service Admin Implementation (1.3.1)
[ 120] [Active  ] [ ] [    ] [   80] CXF dOSGi Topology Manager (1.3.1)
[ 121] [Active  ] [ ] [    ] [   80] Activation 1.1 (1.1)
[ 122] [Active  ] [ ] [    ] [   80] geronimo-annotation_1.0_spec (1.1.1)
[ 123] [Active  ] [ ] [    ] [   80] geronimo-javamail_1.4_spec (1.2)
[ 124] [Active  ] [ ] [    ] [   80] Servlet 3.0 (1.0)
[ 125] [Active  ] [ ] [    ] [   80] Web Services Metadata 2.0 (1.1.3)
[ 126] [Active  ] [ ] [    ] [   80] Jetty::Aggregate::All Server (7.4.2.v20110526)
[ 127] [Active  ] [ ] [    ] [   80] Apache Neethi (3.0.1)
[ 129] [Active  ] [ ] [    ] [   80] Apache Felix File Install (3.1.10)
[ 130] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Bundles::asm (3.3.0.2)
[ 131] [Active  ] [ ] [    ] [   80] Apache ServiceMix Bundles: commons-pool-1.5.4 (1.5.4.1)
[ 132] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Bundles::jaxb-impl (2.1.13.2)
[ 133] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Bundles::joda-time (1.5.2.4)
[ 134] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Bundles::opensaml (2.4.1.1)
[ 135] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Bundles::wsdl4j (1.6.2.5)
[ 136] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Bundles::xmlresolver (1.2.0.4)
[ 137] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Bundles::xmlsec (1.4.5.1)
[ 138] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Specs::JAXB API 2.1 (1.9.0)
[ 139] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Specs::JAXWS API 2.1 (1.9.0)
[ 140] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Specs::JSR-311 API 1.1.1 (1.9.0)
[ 141] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Specs::SAAJ API 1.3 (1.9.0)
[ 142] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Specs::Stax API 1.0 (1.9.0)
[ 143] [Active  ] [ ] [    ] [   80] osgi.enterprise (4.2.0.201003190513)
[ 145] [Active  ] [ ] [    ] [   80] OPS4J Pax Web - Jetty (1.0.3)
[ 146] [Active  ] [ ] [    ] [   80] OPS4J Pax Web - Runtime (1.0.3)
[ 147] [Active  ] [ ] [    ] [   80] OPS4J Pax Web - Service SPI (1.0.3)
[ 148] [Active  ] [ ] [    ] [   80] Spring AOP (3.0.6.RELEASE)
[ 149] [Active  ] [ ] [    ] [   80] Spring ASM (3.0.6.RELEASE)
[ 150] [Active  ] [ ] [    ] [   80] Spring Beans (3.0.6.RELEASE)
[ 151] [Active  ] [ ] [    ] [   80] Spring Context (3.0.6.RELEASE)
[ 152] [Active  ] [ ] [    ] [   80] Spring Core (3.0.6.RELEASE)
[ 153] [Active  ] [ ] [    ] [   80] Spring Expression Language (3.0.6.RELEASE)
[ 154] [Active  ] [ ] [    ] [   80] spring-osgi-core (1.2.1)
[ 155] [Active  ] [ ] [    ] [   80] spring-osgi-extender (1.2.1)
[ 156] [Active  ] [ ] [    ] [   80] spring-osgi-io (1.2.1)
[ 157] [Active  ] [ ] [    ] [   80] Stax2 API (3.1.1)
[ 158] [Active  ] [ ] [    ] [   80] Woodstox XML-processor (4.1.1)
[ 159] [Active  ] [ ] [    ] [   80] XmlSchema Core (2.0.1)
[ 160] [Active  ] [ ] [    ] [   80] ZooKeeper Bundle (3.3.1)
[ 162] [Active  ] [ ] [    ] [   80] Apache Aries Transaction Manager (0.2.0.incubating)
[ 188] [Active  ] [ ] [    ] [   80] OSGi R4 Compendium Bundle (4.1.0)
[ 202] [Active  ] [ ] [    ] [   80] ZooKeeper server configuration bundle (1.3.1)

Note: Make sure zookeeper server bundle NOT installed, which is part of Multi Bundle Delivery, since we are using centralized Zookeeper server; installing server bundle will leads to issues with service distribution.

You may also notice that zookeeper discovery bundle establishes connection to the zookeeper server like,
 

2012-04-24 09:47:25,661 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@604] - Established session 0x136e388a25f0000 with negotiated timeout 4000 for client /127.0.0.1:3480
2012-04-24 09:47:56,004 [myid:] - INFO  [SessionTracker:ZooKeeperServer@334] - Expiring session 0x136dfe81f040001, timeout of 30000ms exceeded
2012-04-24 09:47:56,004 [myid:] - INFO  [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@466] - Processed session termination for sessionid: 0x136dfe81f040001

Once Karaf environment is up and running lets build our interface and implementation for HelloService.


Hello Service Interface
 
package test.helloservice;
public interface HelloService {
    public String sayHello ();
}
 

Maven (pom.xml)

Setup maven build environment to convert interface into OSGi Bundle.

<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>test</groupId>
    <artifactId>test-helloservice-interface</artifactId>
    <packaging>bundle</packaging>
    <name>Test Hello Service Interface Bundle</name>
    <version>1.0-SNAPSHOT</version>
    <parent> <groupId>org.apache.cxf.dosgi</groupId> <artifactId>cxf-dosgi-ri-parent</artifactId>
        <version>1.4-SNAPSHOT</version> <relativePath>parent/pom.xml</relativePath>
        </parent>
    <properties>
        <bundle.import.package>*</bundle.import.package>
        <bundle.export.package>test.helloservice</bundle.export.package>
    </properties>
    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.7</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.easymock</groupId>
            <artifactId>easymock</artifactId>
            <version>2.5.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.easymock</groupId>
            <artifactId>easymockclassextension</artifactId>
            <version>2.5.2</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.felix</groupId>
                <artifactId>maven-bundle-plugin</artifactId>
                <version>2.3.7</version>
                <extensions>true</extensions>
                <configuration>
                    <instructions>
                        <Bundle-Name>Test Hello Service Interface Bundle
                        </Bundle-Name>
                        <Bundle-Description>This bundle contains the implementation of the
                            Test Hello Service Interfaces
                        </Bundle-Description>
                        <Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName>
                        <Import-Package>${bundle.import.package}</Import-Package>
                        <Export-Package>${bundle.export.package}</Export-Package>
                    </instructions>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

Maven Install command from eclipse will build and install the test-helloservice-interface bundle in the .m2 repository. Install the interface bundle in the Karaf environment using the console command.


karaf@root> install mvn:test/test-helloservice-interface/1.0-SNAPSHOT
karaf@root> start 191
karaf@root> list | grep Hello
[ 191] [Active     ] [            ] [       ] [   80] Test Hello Service Interface Bundle (1.0.0.SNAPSHOT)

Hello Service Implementation

package test.helloserviceimpl;
import test.helloservice.HelloService;
public class EngHelloServiceImpl implements HelloService{
    @Override
    public String sayHello() {
        return "Hello";
    }
}

Bundle Activator


public class Activator implements BundleActivator {
    private ServiceRegistration registration;
    @Override
    public void start(BundleContext bc) throws Exception {
        Dictionary props = new Hashtable();
        String host = getHostName();
        int port = getPort();
        props.put("service.exported.interfaces", "*");
        props.put("service.exported.configs", "org.apache.cxf.ws");
        props.put("org.apache.cxf.ws.address", getAddress(host, port));
        props.put("endpoint.id", getAddress(host, port));
        registration = bc.registerService(HelloService.class.getName(),
                new EngHelloServiceImpl(), props);
    }
    private static String getAddress(String host, int port) throws Exception {
        return "http://" + host + ":" + port + "/hello";
    }
    private static String getHostName() {
        try {
            return InetAddress.getLocalHost().getCanonicalHostName();
        } catch (UnknownHostException e) {
            return "localhost";
        }
    }
       private static int getPort() throws IOException {
        return new ServerSocket(0).getLocalPort();
    }
    @Override
    public void stop(BundleContext arg0) throws Exception {
        registration.unregister();
    }
}

Maven (pom.xml)


<project
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion><groupId>test</groupId><artifactId>test-helloservice-implementation</artifactId><packaging>bundle</packaging><name>Test Hello Service Implementation Bundle</name>    <version>1.0-SNAPSHOT</version> <parent>
        <groupId>org.apache.cxf.dosgi</groupId>
        <artifactId>cxf-dosgi-ri-parent</artifactId>
        <version>1.4-SNAPSHOT</version>
        <relativePath>parent/pom.xml</relativePath>
    </parent> <properties>
        <bundle.import.package>*</bundle.import.package>
        <bundle.private.package>test.helloserviceimpl</bundle.private.package>
    </properties> <dependencies> <dependency>
            <groupId>org.apache.felix</groupId>
            <artifactId>org.apache.felix.framework</artifactId>
            <version>${felix.version}</version>
            <exclusions> <exclusion>
                    <groupId>org.apache.felix</groupId>
                    <artifactId>org.osgi.foundation</artifactId>
                </exclusion>  </exclusions>
        </dependency><dependency> <groupId>test</groupId>
            <artifactId>test-helloservice-interface</artifactId>
            <version>${project.version}</version>
        </dependency> </dependencies>
    <build>    <plugins> <plugin>
                <groupId>org.apache.felix</groupId>
                <artifactId>maven-bundle-plugin</artifactId>
                <configuration>  <instructions>
                        <Bundle-Name>Test Hello Service Implementation Bundle</Bundle-Name><Bundle-Description>This bundle contains the implementation of the            Test Hello Service </Bundle-Description>
                        <Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName><Bundle-Activator>test.helloserviceimpl.Activator</Bundle-Activator><Import-Package>${bundle.import.package}</Import-Package>
                        <Private-Package>${bundle.private.package}</Private-Package>    <DynamicImport-Package>org.apache.cxf.dosgi.dsw.qos,org.apache.cxf</DynamicImport-Package>
                    </instructions>
                </configuration> </plugin> </plugins> </build> </project>

Maven Install command from eclipse will build and install the test-helloservice-implementation bundle in the .m2 repository. Install the interface bundle in the Karaf environment using the console command.


karaf@root> install mvn:test/test-helloservice-implementation/1.0-SNAPSHOT
karaf@root> start 197
karaf@root> list | grep Implementation
[ 197] [Active     ] [            ] [       ] [   80] Test Hello Service Implementation Bundle (1.0.0.SNAPSHOT)

Once the implementation bundle activator starts, it registers the EngHelloServiceImpl instance into the zookeeper via CXF discovery service with help of following properties for the service.


props.put("service.exported.interfaces", "*");
props.put("service.exported.configs", "org.apache.cxf.ws");
props.put("org.apache.cxf.ws.address", getAddress(host, port));
props.put("endpoint.id", getAddress(host, port));


To verify the zookeeper service store, start the client and verify the service centralization.

Zookeeper Client Verification


Start the client instance to validate the service registration in the zookeeper server,


zookeeper-3.4.3\bin>zkCli -server 127.0.0.1:2900

2012-04-24 10:41:32,490 [myid:] - INFO  [main-SendThread(localhost:2900):ClientC
nxn$SendThread@846] - Socket connection established to localhost/127.0.0.1:2900,
 initiating session
2012-04-24 10:41:32,583 [myid:] - INFO  [main-SendThread(localhost:2900):ClientC
nxn$SendThread@1175] - Session establishment complete on server localhost/127.0.
0.1:2900, sessionid = 0x136e388a25f0001, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
[zk: 127.0.0.1:2900(CONNECTED) 0]


Get the service endpoint descriptor details using the get command and even by hitting http:/127.0.0.1:4076/hello?wsdl link in the browser.

[zk: 127.0.0.1:2900(CONNECTED) 1] get /osgi/service_registry/test/helloservice/HelloService/localhost#4076##hello
<?xml version="1.0" encoding="UTF-8"?>
<endpoint-descriptions xmlns="http://www.osgi.org/xmlns/rsa/v1.0.0">
  <endpoint-description>
    <property name="endpoint.framework.uuid" value="7761f8fe-edf3-44fd-a5d0-a46902843f1c" />
    <property name="endpoint.id" value="http://localhost:4076/hello" />
    <property name="endpoint.package.version.test.helloservice" value="1.0.0.SNAPSHOT" />
    <property name="endpoint.service.id" value-type="Long" value="217" />
    <property name="objectClass">
      <array>
        <value>test.helloservice.HelloService</value>
      </array>
    </property>
    <property name="org.apache.cxf.ws.address" value="http://localhost:4076/hello" />
    <property name="service.imported" value="true" />
    <property name="service.imported.configs">
      <array>
        <value>org.apache.cxf.ws</value>
      </array>
    </property>
    <property name="service.intents">
      <array>
        <value>SOAP.1_1</value>
        <value>HTTP</value>
        <value>SOAP</value>
      </array>
    </property>
  </endpoint-description>
</endpoint-descriptions>


cZxid = 0x4a
ctime = Tue Apr 24 10:44:45 BST 2012
mZxid = 0x4a
mtime = Tue Apr 24 10:44:45 BST 2012
pZxid = 0x4a

cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x136e388a25f0003
dataLength = 1166
numChildren = 0
 

With this now our EngHelloServiceImpl has been centralized and available for the invocation from the external parties.


Enterprise Service Bus Setup (Instance 2):

Idea is to consume the centralized HelloService via enterprise integration pattern (EIP) or you can say via DSL specification. To achieve we need setup the Apache Servicemix , which can be downloaded from apache site.
Unzip the downloaded servicemix release and start the service mix with following command (make sure JAVA_HOME has been set in the environment)


servicemix-4.4.0\bin>servicemix
____                  _          __  __ _
/ ___|  ___ _ ____   _(_) ___ ___|  \/  (_)_  __
\___ \ / _ \ '__\ \ / / |/ __/ _ \ |\/| | \ \/ /
 ___) |  __/ |   \ V /| | (_|  __/ |  | | |>  <
|____/ \___|_|    \_/ |_|\___\___|_|  |_|_/_/\_\

  Apache ServiceMix (4.4.1)

Hit '' for a list of available commands
and '[cmd] --help' for help on a specific command.
Hit '' or 'osgi:shutdown' to shutdown ServiceMix.

karaf@root>

Load all the DOSGi multi bundles and its dependencies into the Servicemix-4.4.0/deploy directory for hot deployment at end of the deployment following bundles expected to be active within the Servicemix instance. 

Zookeeper needs to make connection to server with port 2900 as we configured in the zoo.cfg file. To do so add into servicemix-install-dir/etc/system.properties with “org.apache.cxf.dosgi.discovery.zookeeper.port = 2900” configuration.


[156] [Active] [] [] [60] Apache CXF Minimal Bundle Jar (2.5.2)
[157] [Active] [] [] [60] ZooKeeper server configuration bundle (1.3.1)
[158] [Active] [] [] [60] CXF dOSGi Remote Service Admin Implementation (1.3.1)
[159] [Active] [] [] [60] CXF dOSGi Topology Manager (1.3.1)
[160] [Active] [] [] [60] CXF Zookeeper-based Discovery Service Bundle (1.3.1)
[161] [Active] [] [] [60] CXF Local Discovery Service Bundle (1.3.1)
[162] [Active] [] [] [60] ZooKeeper Bundle (3.3.1)
[163] [Active] [] [] [60] osgi.enterprise (4.2.0.201003190513)

Note: Make sure zookeeper server bundle NOT installed, which is part of Multi Bundle Deliver, since we are using centralized Zookeeper server installing server bundle will leads to issues with service distribution.
As we have created the Servicemix with CXF-DOSGi environment, now we need to lookup our service, for that we need install HelloService interface bundle in the environment.

karaf@root> install mvn:test/test-helloservice-interface/1.0-SNAPSHOT
karaf@root> start 164
karaf@root> list | grep Hello
[ 164] [Active     ] [            ] [       ] [   60] Test Hello Service Interface Bundle (1.0.0.SNAPSHOT)

As per our goal create EIP CamelContext.xml configuration file using camel and blueprint, which consumes the services. We will create simple routing to access the service deployed in the zookeeper.


<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
http://www.osgi.org/xmlns/blueprint/v1.0.0
http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd">
    <reference id="myService" interface="test.helloservice.HelloService" timeout="20000" availability="optional" />
    <camelContext id="camel" trace="false"
        xmlns="http://camel.apache.org/schema/blueprint">
        <route id="client">
            <from uri="timer://foo?fixedRate=true&amp;period=10000" />
            <bean ref="myService" method="sayHello" />
            <log message=">>> Response from : ${body}" />
        </route>
    </camelContext>
</blueprint>


In the above configuration, originator endpoint will be timer and target endpoint will be our service. Periodically we will invoke the service method. This blueprint xml will be converted into the OSGi bundle on deployment, camelContext lookup the service from the zookeeper via DOSGi discovery bundle.


Copy the CamelContext.xml file into Servicemix hot deployment directory Servicemix-4.4.0\delpoy. You can notice bundle getting created in the Servicemix-karaf console and started.


karaf@root> list | grep Camel

[ 165] [Active     ] [Created     ] [       ] [   60] Camelcontext5.xml (0.0.0)


Immediately in the logs you can notice service ported from zookeeper and been used by the camel for routing and service response printed periodically.

2012-04-24 12:24:15,762 | INFO  | foo              | ReflectionServiceFactoryBean     | ?             ? | 156 - org.apache.cxf.bundle-minimal - 2.5.2 | Creating Service {http://helloservice.test/}HelloService from class test.helloservice.HelloService
2012-04-24 12:24:15,934 | INFO  | foo              | client                           | ?             ? | 91 - org.apache.camel.camel-core - 2.8.4 | >>> Response from : Hello
2012-04-24 12:24:24,668 | INFO  | foo              | client                           | ?             ? | 91 - org.apache.camel.camel-core - 2.8.4 | >>> Response from : Hello
2012-04-24 12:24:34,683 | INFO  | foo              | client                           | ?             ? | 91 - org.apache.camel.camel-core - 2.8.4 | >>> Response from : Hello
2012-04-24 12:24:44,668 | INFO  | foo              | client                           | ?             ? | 91 - org.apache.camel.camel-core - 2.8.4 | >>> Response from : Hello

This “Hello” response end’s article and begins next one. I will keep posted on this blog on further tryouts.