WSO2 ESB Performance Round 7.5

WSO2 has recently performed a comparison study by considering various metrics, and in all the criteria WSO2 ESB has been well performed!! It's extra ordinary speed against to a number of leading open source ESB. Refer WSO2 article on  WSO2 ESB Performance Round 7.5 this for more information.

This comparison was made with WSO2 ESB 4.8.1 against Mule ESB 3.4.0, Talend ESB 5.3.1 and Ulatra ESB 2.0.0. The below shown in the final observations during this.

 

The above results clearly shows that WSO2 ESB performs far more than other leading open source ESBs.

WSO2 Con Asia 2014 in Colombo!

After world wide successful conferences in London and San Francisco, WSO2 brings the same great talks back to Colombo with WSO2 Con Asia 2014! This is the fifth user conference of WSO2 and it will be held March 24-26, 2014 at Waters Edge in Battaramulla, Sri Lanka. WSO2Con Asia 2014 is one of three regional WSO2 user conferences being presented in 2014. The others are WSO2Con Europe 2014, running June 16-18, in Barcelona, Spain, and WSO2Con USA 2014, scheduled for October 27-29 in San Francisco, California. To learn more about all three events, visit http://wso2con.com.

WSO2 yesterday announced the keynote speakers for WSO2Con Asia 2014, including executives from Dialog Axiata PLC/Axiata Group Berhad, Commonwealth Bank, and WSO2. Collectively, these featured speakers will share their vision for the connected business, , as well as introduce case studies, concepts and technologies for achieving successful real-world deployments. More information about the speakers can be found in the official WSO2 Con Asia 2014 speakers' page. 

WSO2 Con Asia is 2 day conference, and also it will be having additional Pre-Conference tutorial session on building enterprise apps, building jaggery js for mobile and web apps, comprehensive cloud strategy for enterprise, API management, Big data with WSO2 CEP and WSO2 BAM, Security, SOA focused enterprise architecture, etc. Throughout the two days of conference, there will be many interesting topics discussed, including cloud, mobile, APIs, social media and open source, is enabling enterprises to create new connections across employees, customers and partners. And also at the end of each day there will be panel discussion to discuss about various perspectives of IT in the current trend.

WSO2Con Asia 2014 is focused on empowering enterprises with the technology and implementation insights to succeed in the era of the connected business. Keynote presenters will explore the emerging technologies that are enabling the connected business, as well as case studies and key concepts around the practical application of these technologies to accelerate development and build a lean and agile connected business environment. The full agenda can be viewed at http://asia14.wso2con.com/agenda.

WSO2 provides the full middle ware open source platform to build, integrate and manage your enterprise APIs, applications, and Web services on-premises, in the cloud, and on mobile devices. And WSO2 Con Asia 2014 is a great opportunity to learn how connected businesses can grow revenue and outperform peers by increasing customer engagement, enhancing productivity, and seizing market opportunity.  Have fun being in WSO2 Con Asia 2014, and expand your knowledge!

Broken pipe exception when connecting to cassandra

 Recently I required to WSO2 BAM receiver with high load. And during that I experienced the below exception.

[2013-03-04 15:56:43,010] ERROR {me.prettyprint.cassandra.connection.client.HThriftClient} -  Could not flush transport (to be expected if the pool is shutting down) in close for client: CassandraClient<cassandra03:9170-1581>
org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe
    at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:147)
    at org.apache.thrift.transport.TFramedTransport.flush(TFramedTransport.java:156)
    at me.prettyprint.cassandra.connection.client.HThriftClient.close(HThriftClient.java:98)
    at me.prettyprint.cassandra.connection.client.HThriftClient.close(HThriftClient.java:26)
    at me.prettyprint.cassandra.connection.HConnectionManager.closeClient(HConnectionManager.java:323)
    at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:272)
    at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecuteOperation(ExecutingKeyspace.java:97)
    at me.prettyprint.cassandra.model.MutatorImpl.execute(MutatorImpl.java:243)
    at org.wso2.carbon.databridge.persistence.cassandra.datastore.CassandraConnector.commit(CassandraConnector.java:177)
    at org.wso2.carbon.databridge.persistence.cassandra.datastore.CassandraConnector.insertEventList(CassandraConnector.java:402)
    at org.wso2.carbon.databridge.datasink.cassandra.subscriber.BAMEventSubscriber.receive(BAMEventSubscriber.java:50)
    at org.wso2.carbon.databridge.core.internal.queue.QueueWorker.run(QueueWorker.java:80)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.SocketException: Broken pipe
    at java.net.SocketOutputStream.socketWrite0(Native Method)
    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
    at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
    at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:145)
    ... 17 more
[2013-03-04 15:56:43,011] ERROR {me.prettyprint.cassandra.connection.HConnectionManager} -  MARK HOST AS DOWN TRIGGERED for host cassandra03(10.157.4.137):9170
[2013-03-04 15:56:43,011] ERROR {me.prettyprint.cassandra.connection.HConnectionManager} -  Pool state on shutdown: <ConcurrentCassandraClientPoolByHost>:


After a doing more research on the cassandra.yaml configuration, i found out changing the below propeties:

thrift_framed_transport_size_in_mb: 15
thrift_max_message_length_in_mb: 16


Increasing the above parameters will solve the problem, ie, you can increase 'thrift_max_message_length_in_mb' to 64 and 'thrift_framed_transport_size_in_mb' to 60, which will help to get rid of the mentioned exception.

Load Balancing Data Publishers and Sending Events to Multiple Receivers

WSO2 BAM/CEP has hight performance thrift based event receiving model, which basically receives the events via TCP. Even though thrift is high performance receiving protocol, load balancing the thrift events is problematic as you need tcp based load balancer rather http based load balancer. Therefore in WSO2 we have added support to have load balancing between Data bridge receivers (ie, WSO2 BAM servers, CEP servers) from the client side, by sending the events in a round robin manner to BAM servers, such that load of events will be balanced between them.

For this we have added a Wrapper class called LoadBalancingDataPublisher, which uses the AsyncDataPublisher in it. It not only load balances the events between the set of servers and also can send same events to some servers. All the capabilities of using the LoadBalancingDataPublisher is provided in BAM 2.2.0 documentation here, which explains the use cases of using the load balancing data publisher.

This provides more fail over handling also with load balancing, which can detect the node failure and stop further publishing for the dead node and it also recognozes the node startup and it starts load balancing the events from that instance.

I'll provide a more detailed description about using LoadDataPublisher to publish events to BAM/CEP in next article.

Non-blocking data publshing for BAM/CEP

You can publish the events to BAM/CEP by using the DataPublisher. AsyncDataPublisher is enhanced version of DataPublisher which incorporates all the constructors and API of of general DataPublisher and send events asynchronously. Thats is in general data publishers, making the connection is synchronous/blocking therefore the network latency might affect the connection time and publishing efficieny. The AsyncDataPublisher connects to the receiver asynchronously and cache and re-use the stream id in efficient manner.

There are mainly three steps involved in creating the AsyncDataPublisher.

1.Create a AsyncDataPublisher Instance with any of available constructors.

Eg: AsyncDataPublisher asyncDataPublisher = new AsyncDataPublisher("url","userName", "password");

2. Add the stream definition json string which you would like to publish via the AsyncDataPublisher. You can add any number of stream definitions and any time you can include that. But make sure before you publish the event for a specific stream definition, you should include it in the AsyncDataPublisher.

Eg: asyncDataPublisher.addStreamDefinition("stream def");

3. Start send events using the AsyncDataPublisher.

Eg: asyncDataPublisher.publish("stream name", "stream version", metaDataObjectArray, correlationDataObjectArray, payLoadDataObjectArray );


This will send events with additional advantage as mentioned above.

BAM Toolboxes - Quick Deployment for your Scenario

I have written BAM toolboxes, which is an installable archive in WSO2 BAM. This provides you a quick deployment of your scenario in WSO2 BAM in few seconds.

WSO2 BAM has importantly three components,

1. Receiving Data/Events and Storage
   - This is the main interface which BAM interacts with other servers. That is you can use WSO2 servers/ any third part servers/application and send the events/data to BAM. I'll include how you can send events to BAM in a separate article.
2. Analytics
   - This component analyzes the data that is received in part-1, and produces some useful information out of your data. Apache Hive and hadoop are used in this segment to analyze the big data and scale according to the requirement.
3. Presentation
   There are attractive dashboards, gadgets, reports can be produced in this segment out of the data you have summarized in step-2.

In reality during the testing stage, you might be writing your own customized artifacts in all three above components. The toolbox provides you the flexibility of combining all three as a single artifact and install in WSO2 BAM, such that all the components as mentioned above can come in to the BAM within few seconds, rather manually setting up the environment in production setup.

And also WSO2 BAM will be continuously releasing new toolboxes which includes different use cases such that you can just install that in WSO2 BAM and try out.

You can see how you can install the toolboxes in WSO2 BAM inhttp://docs.wso2.org/wiki/display/BAM200/Creating+a+Custom+Toolbox http://docs.wso2.org/wiki/display/BAM200/Introduction+to+BAM+Toolbox

And also you can find how you can create your own toolbox in,

http://docs.wso2.org/wiki/display/BAM200/Creating+a+Custom+Toolbox

WSO2 BAM 2.0 - New Begining ...

-->

WSO2 has released WSO2 BAM2.0 with WSO2 CARBON 4.0.1 release on September 8^th 2012.

The WSO2 Business Activity Monitor (WSO2 BAM) is an enterprise-ready, fully-open source, complete solution for aggregating, analyzing and presenting information about business activities. The aggregation refers to collection of data, analysis refers to manipulation of data in order to extract information, and presentation refers to representing this data visually or in other ways such as alerts. The WSO2 BAM architecture reflects this natural flow in its design.

Since all WSO2 products are based on the component-based WSO2 Carbon platform, WSO2 BAM is lean, lightweight and consists of only the required components for efficient functioning. It does not contain unnecessary bulk, unlike many over-bloated, proprietary solutions. WSO2 BAM comprises of only required modules to give the best of performance, scalability and customizability, allowing businesses to achieve time-effective results for their solutions without sacrificing performance or the ability to scale.

The product is available for download at: http://wso2.com/products/business-activity-monitor

The documentation is available at: http://docs.wso2.org/wiki/display/BAM200/WSO2+Business+Activity+Monitor+Documentation

BAM 2.0 has monitoring services, by using the Service Statistics Toolbox found in http://wso2.com/products/business-activity-monitor/2-0-0-toolboxes/. Via this you can monitor request, response, fault count, response time, etc for a hosted services.

WSO2 BAM 2.0.1 which includes attractive dashboards for WSO2 ESB monitoring will be released soon.

Useful commands in OSGI

OSGI framework enables to build true modular system in java. It defines a way to create true bundles and a way those communicate at run time. Here generally the modules are called as bundles and another advantage is we can install and unistall a bundle in the runtime without restarting the JVM.

Here the unit of information hiding is package in a bundle, ie there can be certain packages made visible to other bundles in the runtime and certain can he hidden.

This post is mainly targeted to work an debug wso2 server. Therefore in the run time there is requirement interact with OSGI framework. I'm listing some useful commands here.

1. Wso2server.sh -DosgiConsole
   To enable the OSGI console in server startup

2. install file:<path to file>
   To install a bundle in the run time
   
   
3. ss <some_part_of_bundle_name>
   List the bundle state and bundle id with bundle symbolic name
   
   
4. start <bundle id>
   when you just run the install command, it won't start the bundle. For that you should execute start command also. You can notice this in the ss command.
   
   
5. stop <bunle id>
   This command will put the bundle from ACTIVE to INSTALLED state.
   
   
6. uninstall <bundle_id>
   This will uninstall the bundle from the OSGI framework
   
   
7. packages <package_name>
   You can see the packages visible to the OSGI framework with its versions and exporting bundles.
   Eg: packages org.wso2.foo.bar

8. b <bundle id>
   lists the bundle information, such as exported and imported packages. If you have same package exported by multiple bundles with different versions, then you can view what version of package being used in the run time in the specified bundle id.
   
   
9. diag <bundle id>
   This command lists about the status of the bundle and lists if any unresolved packages exists for that particular bundle.

10. ls
    list down all the service components found in this runtime along with their component ID and life-cycle status. Here you can see whether a bundle in satisfied state or unsatisfied state. For example if a bundle expects a osgi service, and if it's not available in the osgi at run time the bundle will be unsatisfied sate.
    
11. enable <componentID>
    enabling service components
    
    
12. comp <componentID>
    Use this command to find unsatisfied services of the given component, which prevent it from going it to satisfied state. 
    
    
13. services
    list down all the services in the runtime including services provided by the system bundle. You can use simple filters to narrow down your list.

14. disable <componentID> 
    disabling component  given by the component ID

Hope the above commands helps you to work with osgi framework. Have fun with osgi! :-)

I'm in WSO2..

Soon after I complete B.Sc in Engineering in UoM, I joined WSO2 in December 1^st 2011 as a Software Engineer. A next major chapter unfolds in my life from WSO2. It's really wonderful company to work in Sri Lanka. I was assigned to work in BAM(Business Activity Monitoring) which is one of the most expected product in WSO2. I'll write a separate post about WSO2 BAM soon.

WSO2 is a Open source Company, ie, all WSO2 products are 100% open source and released under Apache Licence Version 2.0. It's the only company which provides the complete open source middleware platform. Fully cloud-native, the WSO2 middleware platform is also the only open source platform-as-a-service for private, public and hybrid cloud deployments available today. WSO2 has nearly 15 different kind of products such as WSO2 BAM (Business Activity Monitoring), WSO2 ESB(Enterprise Service Bus), WSO2 IS (Identity Server), WSO2 DSS (Data Services Server), etc which can provide complete solution to enterprise applications. And the count of products doubles to 30 with PaaS (Platform as a Service), ie all WSO2 products are available in WSO2 Stratos. You can visit to WSO2 from www.wso2.com and contact via www.wso2.com/contact for any queries.

WSO2 uses SOA (Service oriented Architecture) throughout it's platform. The WSO2 Carbon, is the core of all WSO2 products. Each products are composed it's respective features. And by installing the respective features of the product on top of carbon will give a new product. And the all products and platform consists of totally independent components where as it can be installed and uninstalled independently. This is achieved through OSGI (Open Services Gateway Initiative) framework. I'll write a separate post on how you can work with OSGI framework.

WSO2 gives me an opportunity to work with lots of products and technologies. From the day one in WSO2, I'm coming across new things and learning continuously. I'm sure WSO2 is going to a big turning point in my life time!!