What is Dumbo MapReduce in Hadoop

Question

I am trying to run a Dumbo MapReduce command on hadoop cluster but getting the following error.

15/03/02 17:55:28 ERROR streaming.StreamJob: Job not successful. Error: NA 15/03/02 17:55:28 INFO streaming.StreamJob: killJob... Streaming Command Failed!

It seems that I need to provide the path to Hadoop streaming jar. How can I find this and what is Hadoop streaming jar?

My dumbo command is as following:

dumbo start test_dumbo6.py -hadoop /apps/hadoop/ -input /storage/user/jj/json.log -output ipcounts2

Answer 1

Doc on Hadoop Streaming : http://hadoop.apache.org/docs/r1.2.1/streaming.html

Hadoop streaming is a part of hadoop core so should be available in $Hadoop_HOME/lib directory.

This library can be added to your job by adding an additional argument :

-libjar <path_to_jar>