Friday, October 5, 2012

Whats is WSO2 BAM 2.0..


WSO2 BAM is a fully open source Business Activity Monitoring. Recently WSO2 BAM 2.0 has been released which a complete revamp of WSO2 BAM 1.x.

It's targeted to monitor not-only WSO2 servers but also other third party servers and applications and Key performance Indicators (KPI) and also providing high scalable storage of data via Cassandra and high business data analytics capability through Apache Hive & Hadoop. Also it has very high performance of data capture by using thrift. WSO2 BAM is very scalable, such as you can scale individual components of BAM according to your requirement. If you need high performance for data storage you can only scale the Cassandra storage as multi-node cluster, and if you target to do more analytics in your captured data then you can scale and have a hadoop cluster which makes your analytics faster. WSO2 BAM also has attractive pluggable dashboards, where some of them are in-built by default and some of them are released by separate toolboxes.WSO2 BAM enables you to not only use the default dashboards but also you can write your own gadgets or visualizing engines to plug and view the data.

The below diagram shows the architecture of WSO BAM.


  1. Event/Data Receiving
    You can publish any data to BAM via data agents. Already there are data agents available for WSO2 Servers in order to publish the Service Statistics Monitoring, and WSO2 ESB for Mediation Statistics Monitoring and Activity Mediation Monitoring. Similar to this you write a simple java application to publish your third-party events through thrift as described in http://wso2.org/library/articles/2012/07/creating-custom-agents-publish-events-bamcep. And also you can publish the events via REST api to BAM as described in http://docs.wso2.org/wiki/display/BAM200/REST+API.

  2. Data Dumping in Cassandra
    After receiving the events/data in BAM, it immediately flush the data Cassandra database. Cassandra data base provides high scalability and it's very suitable to store huge amount of data with hight performance.

  3. Running business analytics
    Analyzer framework consists of Apache Hive and Hadoop. Hive is a data warehouse system for Hadoop that facilitates easy data summarization, ad-hoc queries, and the analysis of large datasets stored in Hadoop compatible file systems. Hive provides a mechanism to project structure onto this data and query the data using a SQL-like language called HiveQL. At the same time this language also allows traditional map/reduce programmers to plug in their custom mappers and reducers when it is inconvenient or inefficient to express this logic in HiveQL.
You can access the captured data which is stored in Cassandra and run the analytics on those data via writing the hive scripts. You can find sample hive scrips on WSO BAM samples in http://docs.wso2.org/wiki/display/BAM200/Samples. and more hive related queries can be found in https://cwiki.apache.org/confluence/display/Hive/Tutorial.

  1. Storing the result in JDBC storage
    After running the business analytics in you captured big data, you can store the summarized results in JDBC storage such as MySQL, H2 Database, Oracle, etc. This facilitates to plug any external reporting engine or preferred dashboard gadgets. In WSO2 BAM samples we have written the summarized data into H2 database as it's the default database used. You can find more details of writing the results in the JDBC storage in http://kasunweranga.blogspot.com/2012/06/jdbc-storage-handler-for-hive.html.

  1. Visualization
    Visualization of the data done by WSO2 Jaggery Framework . This data also can be visualized via writing google gadgets also. You can see the sample gadgets created in WSO2 KPI Monitoring sample in http://docs.wso2.org/wiki/display/BAM200/KPI+Monitoring+Sample. And also there are dashboards which includes specially designed gadgets for Service statistics Monitoring and WSO2 ESB montitoring. 
There is already available reporting using the jasper reporting available in http://docs.wso2.org/wiki/display/BAM200/Reports.

These all phases of WSO2 BAM provides to privilege of monitoring your own data and visualize your business aspects.


Enjoy WSO2 BAM!! :)




2 comments:

  1. Love the blog....

    However, do u have any info about how to hook up jaggery gaghets to the data being sent in....

    ReplyDelete
  2. After writing the summarised data into the JDBC storage, you can just connect to RDBMS (eg: h2, MySQL, etc) as mentioned in http://jaggeryjs.org/apidocs/rdb.jag. And you can query the summarised JDBC table using usual SQL queries.

    ReplyDelete