hadoop fs -test example

To test if file or directory exists in HDFS

If try to use normal shell syntax like

           if [ `hadoop fs -test -d /dev/pathToTest` -ne 0 ]; then

               hadoop fs -mkdir /dev/pathToTest

               echo “Creating  directory”

           fi

Then it does not works.

Reason being the result of the test is reported in the shell exit code, not as a textual output

from the command, just like the UNIX /usr/bin/test command.

The correct usage is given as below

if hadoop fs -test –d /dev/pathToTest ; then

                                echo "Directory  exists"

else

                                hadoop fs -mkdir /dev/pathToTest

                                 echo “Creating  directory”

fi

if hadoop fs -test –e /dev/pathToTest/file.html ; then

                                echo "File  exists"

else

                                echo “File does not exists ”

fi

For more details about HDFS commands please see Hadoop command guide at our favorite website Apache 

http://hadoop.apache.org/docs/r1.1.0/file_system_shell.html

Happy Hadooping :)