Hadoop on AIX

After hours of juggling with my effort to install Hadoop (CDH4) on AIX i concluded that it cannot be done easily. (I know some other flavors say they can)

Sorry for you , if you came after Google search in hope to find something.

I talked with Cloudera also and they said NO :(

My situation was that i wanted to install Hadoop Client on AIX system so that data can be sent from AIX to Hadoop cluster using simple Hadoop put commands.

I installed Java ( IBM JDK 1.6 on the AIX system)

Downloaded the CDH4 tar ball onto system

Set the environment variables

And when i did

$hadoop fs –ls /

I got message as

ls : failure to login

First i thought that its due to fact that hdfs is not allowing me to login to cluster , so i created the user with same name by which i was logged into AIX into Hadoop cluster.

After that also i got the same error.

I thought to ask Cloudera as the current Client had enterprise support :) and i raised query with them

They said IBM JDK don't support some Hadoop specific login mechanism and hence we cannot install Hadoop Client onto AIX

So my next job is to introduce design change and see how to get data inside cluster from AIX machine

• WebHDFS
• HttpFS
• NFSProxy

The above are some of the options which i can propose for the client.

What are your experiences in installing Hadoop on AIX , please do share.

Thanks for reading :)