4PM: Resource Utilization

One of biggest challenge being delivery manager at onsite will be tracking resource utilization for both  offshore & onsite resources and update client accordingly to get new projects. Monitoring resource utilization has lot process and dependencies to get accurate utilization, my purpose is to update my 4PM portal with simple screens to capture the resource planning once the project has been approved.

The dynamic portlets in my dashboard will convert them into nice charts/grids for my tracking and make my job easy.

Utilization Capture Screen

In this screen resource utilization will be captured based on the projects and its account, year and type. Once the utilization are recorded in the database and it can be converted into the dashboard view.

Dynamic Portlet

Dynamic portlet is based on database ORM and condition processing logic which will help to render the chart in the portlet. In the dynamic portlet configuration we can do following,

-          Select the fields required for the rendering

-          Apply the condition for the rendering

-          Get the view

Portlet Configuration – Field Selection

For this portlet juneMonth, julyMonth, augustMonth, sepMonth & type fields has been selected from the “Utilization” table and abbreviation field has been select from “Account” table. These tables are not directly related, but their indirectly related like,

Utilization –[has foreign key relationship]-Project-[has foreign key relationship]-Account

Dynamic portlet module in the 4PM project understands this relationship and prepares the dynamic query to construct the required view.

Portlet Configuration – Field Conditions

For this portlet following conditions to be applied on the selected fields,

-          Sum all juneMonth column values in the table – which means summing up the utilization for all projects.

-          Sum all JulyMonth column values in the table – which means summing up the utilization for all projects.

-          << And same for all selected months >>

-          4PM maintain separate utilization sheet for Onsite and Offshore, pick the rows where the “type” has been marked as "offshore"

-          4PM maintains utilization sheet for many accounts, we need to pick the rows which applies to “test” account.

Lot of left/inner joins queries required to meet this combination. Dynamic portlet module in the 4PM prepares all the combinations and generates database query. In the configuration option, we can simply select the condition type for field like “Chart : Sum”, “Chart : Aggregate”, “Exact : Offshore", etc (custom commands). On successful configuration we get following wonderful portlet view.

Apache Distributed OSGi

OSGi is very famous technology in the standalone applications like eclipse due to its dynamic class loader and interface implementation concepts. OSGi has released enterprise specification and apache provided implementation which CXF-DOSGi. My idea is to create a highly distributed SOA environment and with help of open source technologies and my target architecture is something like below,

Instance 1:  Contains a standalone HelloService implementation
Instance 2:  Enterprise Service Bus setup, with Camel (Enterprise Integration Pattern) configuration.
Instance 3:  Zookeeper Server - Centralized service registry.

Zookeeper Server Setup (Instance 3):

Download zookeeper from apache site and which will be used as centralized service registry for all services. All stored services will be used or accessed by the distributed applications. 

Zookeeper is part of Hadoop project and maintains the services in the cluster of servers. Unzip the zookeeper in any directory and create zoo.cfg file in the [zookeeper-dir]/conf with following configuration.

 
tickTime=2000
initLimit=10
syncLimit=5
dataDir=c:\zookeeper
clientPort=2900

Clientport determines the port in which DOSGi service will use to store the service and lookup the service.  Then need to start the zookeeper server and listen for the client connections, in the bin directory execute following command,

> zkServer

You notice logs for creating new zookeeper snapshots. In this setup we used minimal zookeeper configuration not used distributed server setup.

Hello Service Implementation Setup (Instance 1):

Like to have a Hello Service Interface with “sayHello” method and convert that interface into OSGi bundle and deploy in the Karaf environment. Karaf is OSGi execution environment with excellent shell interface and command framework. You can download the Karaf from apache site, once downloaded unzip in suitable location. 

Set the JAVA_HOME in the environment (if is not set already), with 1.6 version. Karaf not yet certified with 1.7. Run the Karaf command from the bin directory.

>karaf

If the environment is fine, Karaf console welcomes with following JLine text.

E:\SOA\apache-karaf-2.2.6\bin>karaf
  __ __    ____
    / //_/____ __________ _/ __/
   / ,<  / __ `/ ___/ __ `/ /_
  / /| |/ /_/ / /  / /_/ / __/
 /_/ |_|\__,_/_/   \__,_/_/

  Apache Karaf (2.2.6)

Hit '' for a list of available commands
and '[cmd] --help' for help on a specific command.
Hit '' or 'osgi:shutdown' to shutdown Karaf.
karaf@root>

Install the necessary DOSGi bundles to setup the distributed environment. Download apache CXF-DOSGi multi bundles from the apache site and copy the jars to karaf-dir/deploy directory for hot deployment. 

If all dependencies are resolved following bundles should be running in your Karaf environment. 

Zookeeper needs to make connection to server with port 2900 as we configured in the zoo.cfg file. To do so add into karaf-install-dir/etc/system.properties with “org.apache.cxf.dosgi.discovery.zookeeper.port = 2900” configuration.


[ 108] [Active  ] [ ] [    ] [   80] AOP Alliance API (1.0.0)
[ 109] [Active  ] [ ] [    ] [   80] Apache Commons Logging (1.1.1)
[ 110] [Active  ] [ ] [    ] [   80] Apache Log4J (1.2.15)
[ 111] [Active  ] [ ] [    ] [   80] JDOM DOM Processor (1.1.0)
[ 112] [Active  ] [ ] [    ] [   80] SLF4J API (1.5.10)
[ 113] [Resolved ] [ ] [    ] [   80] SLF4J Jakarta Commons Logging Binding (1.5.10)
[ 114] [Active  ] [Created  ] [    ] [   80] Apache CXF Minimal Bundle Jar (2.5.2)
[ 115] [Active  ] [ ] [    ] [   80] CXF Zookeeper-based Discovery Service Bundle (1.3.1)
[ 118] [Active  ] [ ] [    ] [   80] CXF Local Discovery Service Bundle (1.3.1)
[ 119] [Active  ] [ ] [Started] [   80] CXF dOSGi Remote Service Admin Implementation (1.3.1)
[ 120] [Active  ] [ ] [    ] [   80] CXF dOSGi Topology Manager (1.3.1)
[ 121] [Active  ] [ ] [    ] [   80] Activation 1.1 (1.1)
[ 122] [Active  ] [ ] [    ] [   80] geronimo-annotation_1.0_spec (1.1.1)
[ 123] [Active  ] [ ] [    ] [   80] geronimo-javamail_1.4_spec (1.2)
[ 124] [Active  ] [ ] [    ] [   80] Servlet 3.0 (1.0)
[ 125] [Active  ] [ ] [    ] [   80] Web Services Metadata 2.0 (1.1.3)
[ 126] [Active  ] [ ] [    ] [   80] Jetty::Aggregate::All Server (7.4.2.v20110526)
[ 127] [Active  ] [ ] [    ] [   80] Apache Neethi (3.0.1)
[ 129] [Active  ] [ ] [    ] [   80] Apache Felix File Install (3.1.10)
[ 130] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Bundles::asm (3.3.0.2)
[ 131] [Active  ] [ ] [    ] [   80] Apache ServiceMix Bundles: commons-pool-1.5.4 (1.5.4.1)
[ 132] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Bundles::jaxb-impl (2.1.13.2)
[ 133] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Bundles::joda-time (1.5.2.4)
[ 134] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Bundles::opensaml (2.4.1.1)
[ 135] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Bundles::wsdl4j (1.6.2.5)
[ 136] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Bundles::xmlresolver (1.2.0.4)
[ 137] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Bundles::xmlsec (1.4.5.1)
[ 138] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Specs::JAXB API 2.1 (1.9.0)
[ 139] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Specs::JAXWS API 2.1 (1.9.0)
[ 140] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Specs::JSR-311 API 1.1.1 (1.9.0)
[ 141] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Specs::SAAJ API 1.3 (1.9.0)
[ 142] [Active  ] [ ] [    ] [   80] Apache ServiceMix::Specs::Stax API 1.0 (1.9.0)
[ 143] [Active  ] [ ] [    ] [   80] osgi.enterprise (4.2.0.201003190513)
[ 145] [Active  ] [ ] [    ] [   80] OPS4J Pax Web - Jetty (1.0.3)
[ 146] [Active  ] [ ] [    ] [   80] OPS4J Pax Web - Runtime (1.0.3)
[ 147] [Active  ] [ ] [    ] [   80] OPS4J Pax Web - Service SPI (1.0.3)
[ 148] [Active  ] [ ] [    ] [   80] Spring AOP (3.0.6.RELEASE)
[ 149] [Active  ] [ ] [    ] [   80] Spring ASM (3.0.6.RELEASE)
[ 150] [Active  ] [ ] [    ] [   80] Spring Beans (3.0.6.RELEASE)
[ 151] [Active  ] [ ] [    ] [   80] Spring Context (3.0.6.RELEASE)
[ 152] [Active  ] [ ] [    ] [   80] Spring Core (3.0.6.RELEASE)
[ 153] [Active  ] [ ] [    ] [   80] Spring Expression Language (3.0.6.RELEASE)
[ 154] [Active  ] [ ] [    ] [   80] spring-osgi-core (1.2.1)
[ 155] [Active  ] [ ] [    ] [   80] spring-osgi-extender (1.2.1)
[ 156] [Active  ] [ ] [    ] [   80] spring-osgi-io (1.2.1)
[ 157] [Active  ] [ ] [    ] [   80] Stax2 API (3.1.1)
[ 158] [Active  ] [ ] [    ] [   80] Woodstox XML-processor (4.1.1)
[ 159] [Active  ] [ ] [    ] [   80] XmlSchema Core (2.0.1)
[ 160] [Active  ] [ ] [    ] [   80] ZooKeeper Bundle (3.3.1)
[ 162] [Active  ] [ ] [    ] [   80] Apache Aries Transaction Manager (0.2.0.incubating)
[ 188] [Active  ] [ ] [    ] [   80] OSGi R4 Compendium Bundle (4.1.0)
[ 202] [Active  ] [ ] [    ] [   80] ZooKeeper server configuration bundle (1.3.1)

Note: Make sure zookeeper server bundle NOT installed, which is part of Multi Bundle Delivery, since we are using centralized Zookeeper server; installing server bundle will leads to issues with service distribution.

You may also notice that zookeeper discovery bundle establishes connection to the zookeeper server like,
 

2012-04-24 09:47:25,661 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@604] - Established session 0x136e388a25f0000 with negotiated timeout 4000 for client /127.0.0.1:3480
2012-04-24 09:47:56,004 [myid:] - INFO  [SessionTracker:ZooKeeperServer@334] - Expiring session 0x136dfe81f040001, timeout of 30000ms exceeded
2012-04-24 09:47:56,004 [myid:] - INFO  [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@466] - Processed session termination for sessionid: 0x136dfe81f040001

Once Karaf environment is up and running lets build our interface and implementation for HelloService.


Hello Service Interface
 
package test.helloservice;
public interface HelloService {
    public String sayHello ();
}
 

Maven (pom.xml)

Setup maven build environment to convert interface into OSGi Bundle.

<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>test</groupId>
    <artifactId>test-helloservice-interface</artifactId>
    <packaging>bundle</packaging>
    <name>Test Hello Service Interface Bundle</name>
    <version>1.0-SNAPSHOT</version>
    <parent> <groupId>org.apache.cxf.dosgi</groupId> <artifactId>cxf-dosgi-ri-parent</artifactId>
        <version>1.4-SNAPSHOT</version> <relativePath>parent/pom.xml</relativePath>
        </parent>
    <properties>
        <bundle.import.package>*</bundle.import.package>
        <bundle.export.package>test.helloservice</bundle.export.package>
    </properties>
    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.7</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.easymock</groupId>
            <artifactId>easymock</artifactId>
            <version>2.5.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.easymock</groupId>
            <artifactId>easymockclassextension</artifactId>
            <version>2.5.2</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.felix</groupId>
                <artifactId>maven-bundle-plugin</artifactId>
                <version>2.3.7</version>
                <extensions>true</extensions>
                <configuration>
                    <instructions>
                        <Bundle-Name>Test Hello Service Interface Bundle
                        </Bundle-Name>
                        <Bundle-Description>This bundle contains the implementation of the
                            Test Hello Service Interfaces
                        </Bundle-Description>
                        <Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName>
                        <Import-Package>${bundle.import.package}</Import-Package>
                        <Export-Package>${bundle.export.package}</Export-Package>
                    </instructions>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

Maven Install command from eclipse will build and install the test-helloservice-interface bundle in the .m2 repository. Install the interface bundle in the Karaf environment using the console command.


karaf@root> install mvn:test/test-helloservice-interface/1.0-SNAPSHOT
karaf@root> start 191
karaf@root> list | grep Hello
[ 191] [Active     ] [            ] [       ] [   80] Test Hello Service Interface Bundle (1.0.0.SNAPSHOT)

Hello Service Implementation

package test.helloserviceimpl;
import test.helloservice.HelloService;
public class EngHelloServiceImpl implements HelloService{
    @Override
    public String sayHello() {
        return "Hello";
    }
}

Bundle Activator


public class Activator implements BundleActivator {
    private ServiceRegistration registration;
    @Override
    public void start(BundleContext bc) throws Exception {
        Dictionary props = new Hashtable();
        String host = getHostName();
        int port = getPort();
        props.put("service.exported.interfaces", "*");
        props.put("service.exported.configs", "org.apache.cxf.ws");
        props.put("org.apache.cxf.ws.address", getAddress(host, port));
        props.put("endpoint.id", getAddress(host, port));
        registration = bc.registerService(HelloService.class.getName(),
                new EngHelloServiceImpl(), props);
    }
    private static String getAddress(String host, int port) throws Exception {
        return "http://" + host + ":" + port + "/hello";
    }
    private static String getHostName() {
        try {
            return InetAddress.getLocalHost().getCanonicalHostName();
        } catch (UnknownHostException e) {
            return "localhost";
        }
    }
       private static int getPort() throws IOException {
        return new ServerSocket(0).getLocalPort();
    }
    @Override
    public void stop(BundleContext arg0) throws Exception {
        registration.unregister();
    }
}

Maven (pom.xml)


<project
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion><groupId>test</groupId><artifactId>test-helloservice-implementation</artifactId><packaging>bundle</packaging><name>Test Hello Service Implementation Bundle</name>    <version>1.0-SNAPSHOT</version> <parent>
        <groupId>org.apache.cxf.dosgi</groupId>
        <artifactId>cxf-dosgi-ri-parent</artifactId>
        <version>1.4-SNAPSHOT</version>
        <relativePath>parent/pom.xml</relativePath>
    </parent> <properties>
        <bundle.import.package>*</bundle.import.package>
        <bundle.private.package>test.helloserviceimpl</bundle.private.package>
    </properties> <dependencies> <dependency>
            <groupId>org.apache.felix</groupId>
            <artifactId>org.apache.felix.framework</artifactId>
            <version>${felix.version}</version>
            <exclusions> <exclusion>
                    <groupId>org.apache.felix</groupId>
                    <artifactId>org.osgi.foundation</artifactId>
                </exclusion>  </exclusions>
        </dependency><dependency> <groupId>test</groupId>
            <artifactId>test-helloservice-interface</artifactId>
            <version>${project.version}</version>
        </dependency> </dependencies>
    <build>    <plugins> <plugin>
                <groupId>org.apache.felix</groupId>
                <artifactId>maven-bundle-plugin</artifactId>
                <configuration>  <instructions>
                        <Bundle-Name>Test Hello Service Implementation Bundle</Bundle-Name><Bundle-Description>This bundle contains the implementation of the            Test Hello Service </Bundle-Description>
                        <Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName><Bundle-Activator>test.helloserviceimpl.Activator</Bundle-Activator><Import-Package>${bundle.import.package}</Import-Package>
                        <Private-Package>${bundle.private.package}</Private-Package>    <DynamicImport-Package>org.apache.cxf.dosgi.dsw.qos,org.apache.cxf</DynamicImport-Package>
                    </instructions>
                </configuration> </plugin> </plugins> </build> </project>

Maven Install command from eclipse will build and install the test-helloservice-implementation bundle in the .m2 repository. Install the interface bundle in the Karaf environment using the console command.


karaf@root> install mvn:test/test-helloservice-implementation/1.0-SNAPSHOT
karaf@root> start 197
karaf@root> list | grep Implementation
[ 197] [Active     ] [            ] [       ] [   80] Test Hello Service Implementation Bundle (1.0.0.SNAPSHOT)

Once the implementation bundle activator starts, it registers the EngHelloServiceImpl instance into the zookeeper via CXF discovery service with help of following properties for the service.


props.put("service.exported.interfaces", "*");
props.put("service.exported.configs", "org.apache.cxf.ws");
props.put("org.apache.cxf.ws.address", getAddress(host, port));
props.put("endpoint.id", getAddress(host, port));


To verify the zookeeper service store, start the client and verify the service centralization.

Zookeeper Client Verification


Start the client instance to validate the service registration in the zookeeper server,


zookeeper-3.4.3\bin>zkCli -server 127.0.0.1:2900

2012-04-24 10:41:32,490 [myid:] - INFO  [main-SendThread(localhost:2900):ClientC
nxn$SendThread@846] - Socket connection established to localhost/127.0.0.1:2900,
 initiating session
2012-04-24 10:41:32,583 [myid:] - INFO  [main-SendThread(localhost:2900):ClientC
nxn$SendThread@1175] - Session establishment complete on server localhost/127.0.
0.1:2900, sessionid = 0x136e388a25f0001, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
[zk: 127.0.0.1:2900(CONNECTED) 0]


Get the service endpoint descriptor details using the get command and even by hitting http:/127.0.0.1:4076/hello?wsdl link in the browser.

[zk: 127.0.0.1:2900(CONNECTED) 1] get /osgi/service_registry/test/helloservice/HelloService/localhost#4076##hello
<?xml version="1.0" encoding="UTF-8"?>
<endpoint-descriptions xmlns="http://www.osgi.org/xmlns/rsa/v1.0.0">
  <endpoint-description>
    <property name="endpoint.framework.uuid" value="7761f8fe-edf3-44fd-a5d0-a46902843f1c" />
    <property name="endpoint.id" value="http://localhost:4076/hello" />
    <property name="endpoint.package.version.test.helloservice" value="1.0.0.SNAPSHOT" />
    <property name="endpoint.service.id" value-type="Long" value="217" />
    <property name="objectClass">
      <array>
        <value>test.helloservice.HelloService</value>
      </array>
    </property>
    <property name="org.apache.cxf.ws.address" value="http://localhost:4076/hello" />
    <property name="service.imported" value="true" />
    <property name="service.imported.configs">
      <array>
        <value>org.apache.cxf.ws</value>
      </array>
    </property>
    <property name="service.intents">
      <array>
        <value>SOAP.1_1</value>
        <value>HTTP</value>
        <value>SOAP</value>
      </array>
    </property>
  </endpoint-description>
</endpoint-descriptions>


cZxid = 0x4a
ctime = Tue Apr 24 10:44:45 BST 2012
mZxid = 0x4a
mtime = Tue Apr 24 10:44:45 BST 2012
pZxid = 0x4a

cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x136e388a25f0003
dataLength = 1166
numChildren = 0
 

With this now our EngHelloServiceImpl has been centralized and available for the invocation from the external parties.


Enterprise Service Bus Setup (Instance 2):

Idea is to consume the centralized HelloService via enterprise integration pattern (EIP) or you can say via DSL specification. To achieve we need setup the Apache Servicemix , which can be downloaded from apache site.
Unzip the downloaded servicemix release and start the service mix with following command (make sure JAVA_HOME has been set in the environment)


servicemix-4.4.0\bin>servicemix
____                  _          __  __ _
/ ___|  ___ _ ____   _(_) ___ ___|  \/  (_)_  __
\___ \ / _ \ '__\ \ / / |/ __/ _ \ |\/| | \ \/ /
 ___) |  __/ |   \ V /| | (_|  __/ |  | | |>  <
|____/ \___|_|    \_/ |_|\___\___|_|  |_|_/_/\_\

  Apache ServiceMix (4.4.1)

Hit '' for a list of available commands
and '[cmd] --help' for help on a specific command.
Hit '' or 'osgi:shutdown' to shutdown ServiceMix.

karaf@root>

Load all the DOSGi multi bundles and its dependencies into the Servicemix-4.4.0/deploy directory for hot deployment at end of the deployment following bundles expected to be active within the Servicemix instance. 

Zookeeper needs to make connection to server with port 2900 as we configured in the zoo.cfg file. To do so add into servicemix-install-dir/etc/system.properties with “org.apache.cxf.dosgi.discovery.zookeeper.port = 2900” configuration.


[156] [Active] [] [] [60] Apache CXF Minimal Bundle Jar (2.5.2)
[157] [Active] [] [] [60] ZooKeeper server configuration bundle (1.3.1)
[158] [Active] [] [] [60] CXF dOSGi Remote Service Admin Implementation (1.3.1)
[159] [Active] [] [] [60] CXF dOSGi Topology Manager (1.3.1)
[160] [Active] [] [] [60] CXF Zookeeper-based Discovery Service Bundle (1.3.1)
[161] [Active] [] [] [60] CXF Local Discovery Service Bundle (1.3.1)
[162] [Active] [] [] [60] ZooKeeper Bundle (3.3.1)
[163] [Active] [] [] [60] osgi.enterprise (4.2.0.201003190513)

Note: Make sure zookeeper server bundle NOT installed, which is part of Multi Bundle Deliver, since we are using centralized Zookeeper server installing server bundle will leads to issues with service distribution.
As we have created the Servicemix with CXF-DOSGi environment, now we need to lookup our service, for that we need install HelloService interface bundle in the environment.

karaf@root> install mvn:test/test-helloservice-interface/1.0-SNAPSHOT
karaf@root> start 164
karaf@root> list | grep Hello
[ 164] [Active     ] [            ] [       ] [   60] Test Hello Service Interface Bundle (1.0.0.SNAPSHOT)

As per our goal create EIP CamelContext.xml configuration file using camel and blueprint, which consumes the services. We will create simple routing to access the service deployed in the zookeeper.


<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
http://www.osgi.org/xmlns/blueprint/v1.0.0
http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd">
    <reference id="myService" interface="test.helloservice.HelloService" timeout="20000" availability="optional" />
    <camelContext id="camel" trace="false"
        xmlns="http://camel.apache.org/schema/blueprint">
        <route id="client">
            <from uri="timer://foo?fixedRate=true&amp;period=10000" />
            <bean ref="myService" method="sayHello" />
            <log message=">>> Response from : ${body}" />
        </route>
    </camelContext>
</blueprint>


In the above configuration, originator endpoint will be timer and target endpoint will be our service. Periodically we will invoke the service method. This blueprint xml will be converted into the OSGi bundle on deployment, camelContext lookup the service from the zookeeper via DOSGi discovery bundle.


Copy the CamelContext.xml file into Servicemix hot deployment directory Servicemix-4.4.0\delpoy. You can notice bundle getting created in the Servicemix-karaf console and started.


karaf@root> list | grep Camel

[ 165] [Active     ] [Created     ] [       ] [   60] Camelcontext5.xml (0.0.0)


Immediately in the logs you can notice service ported from zookeeper and been used by the camel for routing and service response printed periodically.

2012-04-24 12:24:15,762 | INFO  | foo              | ReflectionServiceFactoryBean     | ?             ? | 156 - org.apache.cxf.bundle-minimal - 2.5.2 | Creating Service {http://helloservice.test/}HelloService from class test.helloservice.HelloService
2012-04-24 12:24:15,934 | INFO  | foo              | client                           | ?             ? | 91 - org.apache.camel.camel-core - 2.8.4 | >>> Response from : Hello
2012-04-24 12:24:24,668 | INFO  | foo              | client                           | ?             ? | 91 - org.apache.camel.camel-core - 2.8.4 | >>> Response from : Hello
2012-04-24 12:24:34,683 | INFO  | foo              | client                           | ?             ? | 91 - org.apache.camel.camel-core - 2.8.4 | >>> Response from : Hello
2012-04-24 12:24:44,668 | INFO  | foo              | client                           | ?             ? | 91 - org.apache.camel.camel-core - 2.8.4 | >>> Response from : Hello

This “Hello” response end’s article and begins next one. I will keep posted on this blog on further tryouts.

  

4PM (For Project Managers)

Suddenly my organization asked to do delivery management activates for an account, technical guy doing management activities is like a hell. Lots and lots of excel sheet and need to submit lot of reports to senior management. Created a portal to solve my day-to-day management activities. Used simple technologies like Django, Dojo & Postgres SQL (DDP), lots of opensource javascript frameworks to solve my portal requirements.

Of course following are my initial technical requirements,

Multiple Portlets : Portlets should be each of its own and do it activities.

Easy for managers : Managers should feels comfortable in populating the details and operate the portlet.

Database ORM : Able to mix and match the database fields with selection conditions to generate view. Like a Pivot table in excel sheet.

Quick Search & Export to excel, mail and ppt format.

Synchronizing with dropbox : database should be synchronized in the dropbox.

Sharing the portlets with other managers

initial functional requirements,

Project Management : Capture the project proposals, milestones, purchase orders, deliverables items in a simple screen for tracking (no complexity with quality and case tools, my organization already have complex sites to handle those requirements)

Finance Management : Fixed bid and T&M invoice management

Process Management : To Design a process workflow for any activity (Under minmap)

Currently portal solves my basic requirements and while implementing got lot ideas to enhance the 4PM portal. Hope will get some time to implement those ideas.

ROR - Ruby on Rails

"Ruby-On-Rails" when I heard the title, thought it would be some alien language/framework which is more complex. Got chance to work on proposal to create Catalogue using the PHP, for that proposal started exploring the framework called CakePHP (which is another interesting framework in PHP); which has borrowed ideas from the Ruby-On-Rails (RoR) framework. RoR is an rapid WEB development framework used in http://www.37signals.com/ site, they where developing the their projects using this in-house framework developed by "David Heinemeier Hansson" http://www.loudthinking.com/about.html 37signals employee and in 2003 Rails framework made available to the OpenSource world.


Rails
Rails is a web development framework written in the Ruby language. It is designed to make programming web applications easier by making several assumptions about what every developer needs to get started. It allows you to write less code while accomplishing more than many other languages and frameworks. Long-time Rails developers also report that it makes web application development more fun.

Rails is opinionated software. That is, it assumes that there is a best way to do things, and it’s designed to encourage that best way – and in some cases to discourage alternatives. If you learn “The Rails Way” you’ll probably discover a tremendous increase in productivity. If you persist in bringing old habits from other languages to your Rails development, and trying to use patterns you learned elsewhere, you may have a less happy experience.

The Rails philosophy includes several guiding principles:

* DRY – “Don’t Repeat Yourself” – suggests that writing the same code over and over again is a bad thing.
* Convention Over Configuration – means that Rails makes assumptions about what you want to do and how you’re going to do it, rather than letting you tweak every little thing through endless configuration files.
* REST is the best pattern for web applications – organizing your application around resources and standard HTTP verbs is the fastest way to go.

Rails is organized around the Model, View, Controller architecture, usually just called MVC.

MVC benefits include:

* Isolation of business logic from the user interface
* Ease of keeping code DRY
* Making it clear where different types of code belong for easier maintenance

Models

A model represents the information (data) of the application and the rules to manipulate that data. In the case of Rails, models are primarily used for managing the rules of interaction with a corresponding database table. In most cases, one table in your database will correspond to one model in your application. The bulk of your application’s business logic will be concentrated in the models.

Views

Views represent the user interface of your application. In Rails, views are often HTML files with embedded Ruby code that performs tasks related solely to the presentation of the data. Views handle the job of providing data to the web browser or other tool that is used to make requests from your application.

Controllers

Controllers provide the “glue” between models and views. In Rails, controllers are responsible for processing the incoming requests from the web browser, interrogating the models for data, and passing that data on to the views for presentation.

Playing With Rails

We will try to create a small web application in the Rails framework, Rails are its dependent gems (jar file equivalent in ruby world).

Example Web Application Creation

Follow below steps to create a small blog application with CRUD (Create, Read, Update and Delete) operations.

Language : Ruby
Server : Mongrel
Database : MySQL

Steps

* WebApp Creation
o Login to Server
o Create blog WebApplication

Execute:
[S171015@ctsinnvlvmft1 ~]$ rails blog

Output:

create
create app/controllers
create app/helpers
create app/models
create app/views/layouts
create config/environments
create config/initializers
create config/locales
create db
create doc
create lib
create lib/tasks
create log
create public/images
create public/javascripts
create public/stylesheets
create script/performance
create test/fixtures
create test/functional
create test/integration
create test/performance
create test/unit
create vendor
create vendor/plugins
create tmp/sessions
create tmp/sockets
create tmp/cache
create tmp/pids
create Rakefile
create README
create app/controllers/application_controller.rb
create app/helpers/application_helper.rb
create config/database.yml
create config/routes.rb
create config/locales/en.yml
create config/initializers/backtrace_silencers.rb
create config/initializers/inflections.rb
create config/initializers/mime_types.rb
create config/initializers/new_rails_defaults.rb
create config/initializers/session_store.rb
create config/environment.rb
create config/boot.rb
create config/environments/production.rb
create config/environments/development.rb
create config/environments/test.rb
create script/about
create script/console
create script/dbconsole
create script/destroy
create script/generate
create script/runner
create script/server
create script/plugin
create script/performance/benchmarker
create script/performance/profiler
create test/test_helper.rb
create test/performance/browsing_test.rb
create public/404.html
create public/422.html
create public/500.html
create public/index.html
create public/favicon.ico
create public/robots.txt
create public/images/rails.png
create public/javascripts/prototype.js
create public/javascripts/effects.js
create public/javascripts/dragdrop.js
create public/javascripts/controls.js
create public/javascripts/application.js
create doc/README_FOR_APP
create log/server.log
create log/production.log
create log/development.log
create log/test.log


* Database Configuration
o Modify the config/database.yml configuration

development:
adapter: mysql
encoding: utf8
reconnect: false
database: blog_development
pool: 5
username: S171015
password: password-1
socket: /var/lib/mysql/mysql.sock

test:
adapter: mysql
encoding: utf8
reconnect: false
database: blog_test
pool: 5
username: S171015
password: password-1
socket: /var/lib/mysql/mysql.sock

production:
adapter: mysql
encoding: utf8
reconnect: false
database: blog_production
pool: 5
username: S171015
password: password-1
socket: /var/lib/mysql/mysql.sock


* Create Database

[S171015@ctsinnvlvmft1 blog]$ rake db:create


Above command rake is general-purpose command-runner, which creates the database called 'blog_development.

* Create Resources

Execute:
[S171015@ctsinnvlvmft1 blog]$ script/generate scaffold Post name:string title:string content:text

Output:

exists app/models/
exists app/controllers/
exists app/helpers/
create app/views/posts
exists app/views/layouts/
exists test/functional/
exists test/unit/
create test/unit/helpers/
exists public/stylesheets/
create app/views/posts/index.html.erb
create app/views/posts/show.html.erb
create app/views/posts/new.html.erb
create app/views/posts/edit.html.erb
create app/views/layouts/posts.html.erb
create public/stylesheets/scaffold.css
create app/controllers/posts_controller.rb
create test/functional/posts_controller_test.rb
create app/helpers/posts_helper.rb
create test/unit/helpers/posts_helper_test.rb
route map.resources :posts
dependency model
exists app/models/
exists test/unit/
exists test/fixtures/
create app/models/post.rb
create test/unit/post_test.rb
create test/fixtures/posts.yml
create db/migrate
create db/migrate/20090703064638_create_posts.rb

Command 'script/generate' used to generate files required by the rails, scaffold command will get you up and running quickly, it generates “one size fits all” code for quick development. In this example database table named 'Posts' will be created and associated MVC files for the same.

Controller File

file: app/controller/posts_controller.rb

class PostsController < ApplicationController
# GET /posts
# GET /posts.xml
def index # INDEX Action
...
end
# GET /posts/1
# GET /posts/1.xml
def show # Action to SHOW the post
...
end
# GET /posts/new
# GET /posts/new.xml
def new # Action to ADD new post
...
end
# GET /posts/1/edit
def edit # Action to EDIT post
...
end
# POST /posts
# POST /posts.xml
def create # Action to CREATE post
...
end
# PUT /posts/1
# PUT /posts/1.xml
def update # Action to UPDATE post
...
end
# DELETE /posts/1
# DELETE /posts/1.xml
def destroy # Action to DELETE post
@post = Post.find(params[:id])
...
end


Model File

file: app/models/post.rb

class Post < ActiveRecord::Base
...
end


View Files Contains minimal html code to render the content in the browser.

file: app/views/posts/edit.html.erb
file: app/views/posts/index.html.erb
file: app/views/posts/new.html.erb
file: app/views/posts/show.html.erb

Starting the Server


[S171015@ctsinnvlvmft1 blog]$ script/server
=> Booting Mongrel
=> Rails 2.3.2 application starting on http://localhost:3000
=> Call with -d to detach
=> Ctrl-C to shutdown server




Access the Blog Application

http://localhost:3000/posts



Add new Post


Listing all Posts

DOJO - My old friend

Dojo, I learnt this RIA framework in my earlier career to prepare GUI for one of my project tool. Now framework has evolved lot with stunning features, I decided to use this framework for my new editorial software "Guide For Web - source collection", for which I need User Interface to categorize the websites with screen-shots and reviews.

Dojo (Interface) --> Apache + PHP + CakePHP (Core) --> Postgresql (Backend)

Completed initial step in creating the Dojo view, hope this weekend will integrate with CakePHP to get this up and running.

Dojo View

Step1:1 - ".emu" file format without policy

I want to structure the ".emu" file more simple and easy to process, decided to use the JSON format (as Amazon) and came up with following formats for Buckets and Object. Right now this format is more theoretical, it will be updated iteratively.

Once this format is applied for the Buckets and Objects, planning to include Policy also.


Bucket Format


|- myBucket
|- .emu -> Json file represents bucket properties

{ "bucketname": {
"accessControlList": [
{ "ID": "a9a7b886d6fd24a52fe8ca5bef65f89a64e0193f23000e241bf9b1c61be666e9", "DisplayName":"chriscustomer", "Permission": "FULL_CONTROL"},
{ "ID": "a9a7b886d6fd24a52fe8ca5bef65f89a64e0193f23000e241bf9b1c61be666e9", "DisplayName":"chriscustomer", "Permission": "FULL_CONTROL"},
{ "ID": "a9a7b886d6fd24a52fe8ca5bef65f89a64e0193f23000e241bf9b1c61be666e9", "DisplayName":"chriscustomer", "Permission": "FULL_CONTROL"},
],
"AWSAccessKeyId": "AWS87289288929829kkjkhS88",
"timestamp": "2010-10-02 11:00:00",
"signature": "AAAAAAAAAAAAAAAAAAAAAAAAAAA",
"objectCount": "0"
}
}

Object Format


|- Object1
|- Object2
|- Key1/Object3
|- Key2/Object4
|- ".emu"

{ "bucketname": {
"objects": [
{"object1": {
"key": "Key1",
"metadata": [
{"metadata_obj": { "name": "Content-Type", "value": "text/plain" }},
{"metadata_obj": { "name": "Content-Type", "value": "text/plain" }},
{"metadata_obj": { "name": "Content-Type", "value": "text/plain" }}
],
"data": "1234567",
"contentlength": "5",
"accessControlList": [
{ "ID": "a9a7b886d6fd24a52fe8ca5bef65f89a64e0193f23000e241bf9b1c61be666e9", "DisplayName":"chriscustomer", "Permission": "FULL_CONTROL"},
{ "ID": "a9a7b886d6fd24a52fe8ca5bef65f89a64e0193f23000e241bf9b1c61be666e9", "DisplayName":"chriscustomer", "Permission": "FULL_CONTROL"},
{ "ID": "a9a7b886d6fd24a52fe8ca5bef65f89a64e0193f23000e241bf9b1c61be666e9", "DisplayName":"chriscustomer", "Permission": "FULL_CONTROL"},
],
"AWSAccessKeyId": "AWS87289288929829kkjkhS88",
"timestamp": "2010-10-02 11:00:00",
"signature": "AAAAAAAAAAAAAAAAAAAAAAAAAAA }},
{"object2": {
"key": "Key2",
"metadata": [
{"metadata_obj": { "name": "Content-Type", "value": "text/plain" }},
{"metadata_obj": { "name": "Content-Type", "value": "text/plain" }},
{"metadata_obj": { "name": "Content-Type", "value": "text/plain" }}
],
"data": "1234567",
"contentlength": "5",
"accessControlList": [
{ "ID": "a9a7b886d6fd24a52fe8ca5bef65f89a64e0193f23000e241bf9b1c61be666e9", "DisplayName":"chriscustomer", "Permission": "FULL_CONTROL"},
{ "ID": "a9a7b886d6fd24a52fe8ca5bef65f89a64e0193f23000e241bf9b1c61be666e9", "DisplayName":"chriscustomer", "Permission": "FULL_CONTROL"},
{ "ID": "a9a7b886d6fd24a52fe8ca5bef65f89a64e0193f23000e241bf9b1c61be666e9", "DisplayName":"chriscustomer", "Permission": "FULL_CONTROL"},
],
"AWSAccessKeyId": "AWS87289288929829kkjkhS88",
"timestamp": "2010-10-02 11:00:00",
"signature": "AAAAAAAAAAAAAAAAAAAAAAAAAAA" }},
{"object3": {
"key": "Key3",
"metadata": [
{"metadata_obj": { "name": "Content-Type", "value": "text/plain" }},
{"metadata_obj": { "name": "Content-Type", "value": "text/plain" }},
{"metadata_obj": { "name": "Content-Type", "value": "text/plain" }}
],
"data": "1234567",
"contentlength": "5",
"accessControlList": [
{ "ID": "a9a7b886d6fd24a52fe8ca5bef65f89a64e0193f23000e241bf9b1c61be666e9", "DisplayName":"chriscustomer", "Permission": "FULL_CONTROL"},
{ "ID": "a9a7b886d6fd24a52fe8ca5bef65f89a64e0193f23000e241bf9b1c61be666e9", "DisplayName":"chriscustomer", "Permission": "FULL_CONTROL"},
{ "ID": "a9a7b886d6fd24a52fe8ca5bef65f89a64e0193f23000e241bf9b1c61be666e9", "DisplayName":"chriscustomer", "Permission": "FULL_CONTROL"},
],
"AWSAccessKeyId": "AWS87289288929829kkjkhS88",
"timestamp": "2010-10-02 11:00:00",
"signature": "AAAAAAAAAAAAAAAAAAAAAAAAAAA" }},
{"object4": {
"key": "Key4",
"metadata": [
{"metadata_obj": { "name": "Content-Type", "value": "text/plain" }},
{"metadata_obj": { "name": "Content-Type", "value": "text/plain" }},
{"metadata_obj": { "name": "Content-Type", "value": "text/plain" }}
],
"data": "1234567",
"contentlength": "5",
"accessControlList": [
{ "ID": "a9a7b886d6fd24a52fe8ca5bef65f89a64e0193f23000e241bf9b1c61be666e9", "DisplayName":"chriscustomer", "Permission": "FULL_CONTROL"},
{ "ID": "a9a7b886d6fd24a52fe8ca5bef65f89a64e0193f23000e241bf9b1c61be666e9", "DisplayName":"chriscustomer", "Permission": "FULL_CONTROL"},
{ "ID": "a9a7b886d6fd24a52fe8ca5bef65f89a64e0193f23000e241bf9b1c61be666e9", "DisplayName":"chriscustomer", "Permission": "FULL_CONTROL"},
],
"AWSAccessKeyId": "AWS87289288929829kkjkhS88",
"timestamp": "2010-10-02 11:00:00",
"signature": "AAAAAAAAAAAAAAAAAAAAAAAAAAA" }}
]
}
}


Step1:0: Study - Amazon S3 Emulator

There are lot of ways to implement this emulator and lot of existing implementation also available to do this, but purpose of this emulator implementation is to understand the Amazon S3 operations (which I am going to use in upcoming steps). My only requirement is "it should be identical to the Amazon S3 behaviour", way we access the service and all other functionalities. Intern I want to test with Amazon Java/PHP APIs (sorry I am not interested in .Net)

Downloaded the AmazonS3.wsdl & AmazonS3.xsd and generated the Axis implementation for the same. Initially thought of using database with hibernate to store the buckets and objects, but with this option we cannot access the uploaded objects as Amazon S3 provides. Decided to store the objects in the disk as node file, which web server can access. I have planned to use Apache Commons IO and Apache MINA libraries for the same.

Bucket

- Will be like webapps

- Access control, AWSAccessKeyId, timestamp, etc list will be maintained in ".emu" file. (like .svn not folder, but file)

Example:

<webapps>/myBuket/

Can be accessed from outside like, http://localhost:port/myBuket/

Object

- Key will be name of the object (file)

- Access control, AWSAccessKeyId, timestamp, etc list will be maintained in ".emu" file. (like .svn not folder, but file)

Example:

webapps/myBuket/profile.jpg

Can be accessed from outside like, http://localhost:port/myBuket/profile.jpg

Similarly all other operations supported by the Amazon S3 will be provided.

Next

Step1:1 Complete: Amazon S3 Emulator – Policy Implementation, Step1:2 Complete: Amazon S3 Emulator, Step1:3 Study - Amazon S3 with Hadoop,…..

Step 2:0 Study - Apache Jackrabbit - Amazon S3..