Load Balancing Data Publishers and Sending Events to Multiple Receivers

WSO2 BAM/CEP has hight performance thrift based event receiving model, which basically receives the events via TCP. Even though thrift is high performance receiving protocol, load balancing the thrift events is problematic as you need tcp based load balancer rather http based load balancer. Therefore in WSO2 we have added support to have load balancing between Data bridge receivers (ie, WSO2 BAM servers, CEP servers) from the client side, by sending the events in a round robin manner to BAM servers, such that load of events will be balanced between them.

For this we have added a Wrapper class called LoadBalancingDataPublisher, which uses the AsyncDataPublisher in it. It not only load balances the events between the set of servers and also can send same events to some servers. All the capabilities of using the LoadBalancingDataPublisher is provided in BAM 2.2.0 documentation here, which explains the use cases of using the load balancing data publisher.

This provides more fail over handling also with load balancing, which can detect the node failure and stop further publishing for the dead node and it also recognozes the node startup and it starts load balancing the events from that instance.

I'll provide a more detailed description about using LoadDataPublisher to publish events to BAM/CEP in next article.

Non-blocking data publshing for BAM/CEP

You can publish the events to BAM/CEP by using the DataPublisher. AsyncDataPublisher is enhanced version of DataPublisher which incorporates all the constructors and API of of general DataPublisher and send events asynchronously. Thats is in general data publishers, making the connection is synchronous/blocking therefore the network latency might affect the connection time and publishing efficieny. The AsyncDataPublisher connects to the receiver asynchronously and cache and re-use the stream id in efficient manner.

There are mainly three steps involved in creating the AsyncDataPublisher.

1.Create a AsyncDataPublisher Instance with any of available constructors.

Eg: AsyncDataPublisher asyncDataPublisher = new AsyncDataPublisher("url","userName", "password");

2. Add the stream definition json string which you would like to publish via the AsyncDataPublisher. You can add any number of stream definitions and any time you can include that. But make sure before you publish the event for a specific stream definition, you should include it in the AsyncDataPublisher.

Eg: asyncDataPublisher.addStreamDefinition("stream def");

3. Start send events using the AsyncDataPublisher.

Eg: asyncDataPublisher.publish("stream name", "stream version", metaDataObjectArray, correlationDataObjectArray, payLoadDataObjectArray );


This will send events with additional advantage as mentioned above.