Sunday, January 24, 2016

ClickStream Data Analytics end-to-end

http://hortonworks.com/hadoop-tutorial/how-to-visualize-website-clickstream-data/

Coursera Big Data Training

Affordable training... online from a reputed university. https://www.coursera.org/specializations/big-data?utm_medium=onlineads&utm_campaign=Big+Data&utm_source=fb&nan_pid=1846868993

Tuesday, December 29, 2015

$5 for any PacktPub book


Improve your skills during the holidays... https://www.packtpub.com/all/?search=hdinsight%20essentials

Monday, December 21, 2015

Apache Kylin - Olap for Hadoop

http://kylin.apache.org Ebay technologies open sourced this. http://www.ebaytechblog.com/2014/10/20/announcing-kylin-extreme-olap-engine-for-big-data/ Check out the web interface... http://kylin.apache.org/docs/tutorial/web.html

Sunday, December 13, 2015

Apache Atlas on Data Governance

Apache Atlas is trying to be the Data Governance solution on Hadoop. Their approach has been more about integrating various metadata stores in one place which seems to be lacking in several organizations. It is still in it's early days of development. Currently they have integration with Hive from metadata bridge and Apache Ranger integration for policy enforcements.

Wednesday, October 21, 2015

How to run HDP on Azure

There are few options if you are looking at Azure to host Hadoop

  1. HDInsight - which is Microsoft's flavor of Hadoop (built on top of HDP).  This does provide good separation of storage Azure Blobstore + compute on hardware.
  2. HDP on Azure is new option, where you can get real Hortonworks distribution spun up as VM's.   Each data node is serving data and is managing compute. However in this model you cannot use Azure Blob storage easily.  If you shutdown the cluster, data is gone, unless you script the storage back to Azure blob storage and have the reverse in case you want to bring data back to Hadoop.
  3. Azure is also supporting another model that is useful for screnarios like "OnPremp" Hadoop cluster for day to day operations, but backup to Azure Blob storage.  This utilizes HDFS abstraction.  The demo below will show you how to setup a feed with 2 data paths... primary and secondary.  The secondary is on Azure blobstorage.