How will you submit job as particular user

Question

While running mapreduce job, the jobs are run as the default user. What if I want to run the job as a different user? How can I do that?

Answer 1

You need to create a new user before submitting the Job. Please refer to the below details:

I. Create a New User

Ubuntu: $ sudo adduser –ingroup

Redhat: $ useradd -g

All: $ passwd

II. Change permission of hadoop temp directory
First, change the permission of a directory in HDFS where hadoop stores its temporary data. You will find it in parameter hadoop.tmp.dir in core-site.xml file. Then run this command:

$ hadoop fs –chmod -R 1777 /path/to/hadoop/tmp/directory

III. Give write access to hadoop temp directory for new user

Now we have to give write permissions to hadoop.tmp.dir. Run this command:

$ chmod 777 /path/to/hadoop/tmp/dirctory

IV. Create user home directory in HDFS

$ hadoop fs –mkdir /user/username/

V. Change the ownership of user home directory

The new user will not have permission to run mapreduce jobs. For that, you will have to change ownership:

$ hadoop fs –chown –R username:groupname
Eg: hadoop fs –chown –R username:groupname /user/username/

VI. Now run jobs with new user
Login with new user and run jobs

$ su – newuser
$ bin/hadoop jar pathToJar.jar mainClass input/path output/path