no such file or directory in case of hadoop fs -ls

Question

I have installed and configured Hadoop 2.7.6 for a 12 node cluster. 1 is acting as masternode and other nodes as slavenodes.

I am facing a problem in executing hadoop fs commands. 
hadoop fs -ls command is working fine with HDFS URI. But it gives me error message "ls: `.': No such file or directory" when used without HDFS URI

ubuntu@sam-master:~$ hadoop fs -ls
15/01/30 17:03:49 WARN util.NativeCodeLoader: Unable to load native-hadoop 
ibrary for your platform... using builtin-java classes where applicable
ls: `.': No such file or directory
ubuntu@sam-master:~$ 
Whereas, executing the same command with HDFS URI

ubuntu@sam-master:~$ hadoop fs -ls hdfs://sam-master:50000/
15/01/30 17:14:31 WARN util.NativeCodeLoader: Unable to load native-hadoop      
library for your platform... using builtin-java classes where applicable
Found 3 items
drwxr-xr-x   - ubuntu supergroup          0 2015-01-28 12:07 hdfs://sam-master:50000/hvision-data
-rw-r--r--   2 ubuntu supergroup   15512587 2015-01-28 11:50 hdfs://sam-master:50000/testimage.seq
 drwxr-xr-x   - ubuntu supergroup          0 2015-01-30 17:03 hdfs://sam-master:50000/wrodcount-in
 ubuntu@sam-master:~$ 
I am getting exception in MapReduce program due to this behavior. jarlib is referring to the HDFS file location, whereas, I want jarlib to refer to the jar files stored at the local file system on the Hadoop nodes.

What is happening with me. I don't understand, please help!

Thanks in advance.

Answer 1

The behaviour that you are seeing is expected, let me explain what's going on when you are working with hadoop fs commands.

The exact command's syntax is this: hadoop fs -ls [path]

By default, when you don't specify [path] for the above command, hadoop expands the path to /home/[username] in hdfs; where [username] gets replaced with linux username who is executing the command.

So, when you execute this command:

ubuntu@xadsam-master:~$ hadoop fs -ls
the reason you are seeing the error is ls: '.': No such file or directory because hadoop is looking for this path /home/ubuntu, it seems like this path doesn't exist in hdfs.

The reason why this command:

ubuntu@sam-master:~$ hadoop fs -ls hdfs://101-master:50000/
is working because, you have explicitly specified [path] and is the root of the hdfs. You can also do the same using this:

ubuntu@sam-master:~$ hadoop fs -ls /
which automatically gets evaluated to the root of hdfs.

Hope, this clears the behaviour you are seeing while executing hadoop fs -ls command.

Hence, if you want to specify local file system path use file:/// url scheme.

Hope this will clear all your doubts.