Strata Hadoop World 2014 Conference Speaker Notes and Links

I attended the 2014 Strata Hadoop Conference @ NY which was a great success and increase in participation compared to last year. Spark was a key highlight of the conference from a technology perspective. There are several new products and tools trying to capitalize the predicted 40$ Billon market. Link to slides and videos: http://strataconf.com/stratany2014/public/schedule/proceedings?imm_mid=0c5096&cmp=em-strata-na-info-stny14_thankyou

Hbase and Hive Integration

HBase has been the key database in Hadoop ecosystem providing transactional support enabling real time applications to be built on top of HDFS. Following is good article that describe the roadmap of Hbase and Hive from Hortonworks. This will help and streamline architectures in Hadooop http://hortonworks.com/blog/hbase-hive-better-together/

HiveServer2 architecture

This is a good article on Hive Server2 architecture

HiveServer2 from Schubert Zhang

How to kill a MapReduce job

Very common need, kill a long running job.  following is the syntax

$ hadoop job -kill <job-id>

Usage: hadoop job [GENERIC_OPTIONS] [-submit <job-file>] | [-status <job-id>] | [-counter <job-id> <group-name> <counter-name>] | [-kill <job-id>] | [-events <job-id> <from-event-#> <#-of-events>] | [-history [all] <jobOutputDir>] | [-list [all]] | [-kill-task <task-id>] | [-fail-task <task-id>] | [-set-priority <job-id> <priority>]

Yarn Capacity Schedular

Good article explaining how to use capacity scheduler

https://support.gopivotal.com/hc/en-us/articles/201623853-How-to-configure-queues-using-YARN-capacity-scheduler-xml-

Hadoop I/O

This maybe the next generation of MapReduce.

https://www.google.com/events/io/io14videos

Hadoop Summit 2014 Presentations online

Links to keynote and presentations of Hadoop Summit 2014.  YARN was the focus of this year and MapReduce will soon become legacy:

• http://hadoopsummit.org/san-jose/keynote-day1/
• http://hadoopsummit.org/san-jose/schedule/