Sqoop Got error creating database manager

Error 1

12/02/28 18:47:04 ERROR tool.BaseSqoopTool: Got error creating database manager: java.io.IOException: No manager for connect string: jdbc.mysql://192.168.0.13/hadoopsqoops
at com.cloudera.sqoop.ConnFactory.getManager(ConnFactory.java:119)
at com.cloudera.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:189)
at com.cloudera.sqoop.tool.ImportTool.init(ImportTool.java:81)
at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:411)
at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)


http://stackoverflow.com/questions/8455559/got-error-creating-database-manager-error-in-sqoop-import-query


sqoop import --connect jdbc:mysql://192.168.0.13:3306/hadoopSqoop --username root --password root --table widgets -m 1






Error 3

Host '192.168.0.100' is not allowed to connect to this MySQL server"

Its pure Mysql error related to permissions
Grant user from other host to connect
http://groups.google.com/group/mailing.database.mysql/browse_frm/thread/759d9fff020fce27


Erro 4



Error 5




Error 6

12/02/28 20:26:37 ERROR orm.CompilationManager: Could not rename /tmp/sqoop-hadoop/compile/6d62f2246e8b34f9a6fb09aea4bbfe6c/widgets.java to /home/hadoop/./widgets.java
java.io.IOException: Destination '/home/hadoop/./widgets.java' already exists











hadoop@localhost's password:
localhost: Permission denied (publickey,password).
hadoop@jj-VirtualBox:~$
hadoop@jj-VirtualBox:~$ ^C
hadoop@jj-VirtualBox:~$ chmod go-w $HOME $HOME/.ssh
hadoop@jj-VirtualBox:~$ chmod 600 $HOME/.ssh/authorized_keys
hadoop@jj-VirtualBox:~$ chown `whoami` $HOME/.ssh/authorized_keys
hadoop@jj-VirtualBox:~$ start-dfs.sh
Warning: $HADOOP_HOME is deprecated.

starting namenode, logging to /home/hadoop/software/hadoop-1.0.1/libexec/../logs/hadoop-hadoop-namenode-jj-VirtualBox.out
localhost: starting datanode, logging to /home/hadoop/software/hadoop-1.0.1/libexec/../logs/hadoop-hadoop-datanode-jj-VirtualBox.out
localhost: starting secondarynamenode, logging to /home/hadoop/software/hadoop-1.0.1/libexec/../logs/hadoop-hadoop-secondarynamenode-jj-VirtualBox.out
hadoop@jj-VirtualBox:~$ start-mapred.sh
Warning: $HADOOP_HOME is deprecated.

starting jobtracker, logging to /home/hadoop/software/hadoop-1.0.1/libexec/../logs/hadoop-hadoop-jobtracker-jj-VirtualBox.out
localhost: starting tasktracker, logging to /home/hadoop/software/hadoop-1.0.1/libexec/../logs/hadoop-hadoop-tasktracker-jj-VirtualBox.out
hadoop@jj-VirtualBox:~$ sqoop import --connect jdbc:mysql://192.168.0.13:3306/hadoopSqoop --username root --password root --table widgets -m 1
Warning: /usr/lib/hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: $HADOOP_HOME is deprecated.

12/02/28 20:26:34 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
12/02/28 20:26:34 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
12/02/28 20:26:34 INFO tool.CodeGenTool: Beginning code generation
12/02/28 20:26:34 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `widgets` AS t LIMIT 1
12/02/28 20:26:34 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `widgets` AS t LIMIT 1
12/02/28 20:26:34 INFO orm.CompilationManager: HADOOP_HOME is /home/hadoop/software/hadoop-1.0.1/libexec/..
12/02/28 20:26:37 ERROR orm.CompilationManager: Could not rename /tmp/sqoop-hadoop/compile/6d62f2246e8b34f9a6fb09aea4bbfe6c/widgets.java to /home/hadoop/./widgets.java
java.io.IOException: Destination '/home/hadoop/./widgets.java' already exists
at org.apache.commons.io.FileUtils.moveFile(FileUtils.java:1811)
at com.cloudera.sqoop.orm.CompilationManager.compile(CompilationManager.java:227)
at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:83)
at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:337)
at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:423)
at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)
12/02/28 20:26:37 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/6d62f2246e8b34f9a6fb09aea4bbfe6c/widgets.jar
12/02/28 20:26:37 WARN manager.MySQLManager: It looks like you are importing from mysql.
12/02/28 20:26:37 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
12/02/28 20:26:37 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
12/02/28 20:26:37 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
12/02/28 20:26:37 INFO mapreduce.ImportJobBase: Beginning import of widgets
12/02/28 20:26:37 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `widgets` AS t LIMIT 1

Hadoop SSH asking for password

If once you have setup SSH properly and you see that hadoop is again asking for password while SSH connection then you need to do following

$ chmod go-w $HOME $HOME/.ssh
$ chmod 600 $HOME/.ssh/authorized_keys
$ chown `whoami` $HOME/.ssh/authorized_keys

For detailed instructions on SSH setup for Hadoop , please read this post below

http://allthingshadoop.com/2010/04/20/hadoop-cluster-setup-ssh-key-authentication/

Sqoop Call to localhost/127.0.0.1:8021 failed

Encountered IOException running import job: java.net.ConnectException: Call to localhost/127.0.0.1:8021 failed on connection exception: java.net.ConnectException: Connection refused

Simple indication is that Hadoop is not running , just check your hadoop service

Could not load db driver class: com.mysql.jdbc.Driver Sqoop

While working with Sqoop i got this error

java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver

Solution

MySQL jar is missing , download MySQL java connector jar and save it into sqoop/lib folder

You can download it from http://www.mysql.com/downloads/connector/j/

More details at Solution post
https://groups.google.com/a/cloudera.org/group/sqoop-user/browse_thread/thread/a7075a8557c00007?pli=1

The MySQL JDBC Driver was removed from Sqoop distribution in order to ensure that the default distribution is fully Apache license compliant.

Edit file as root using gedit in Ubuntu

Often we see file as readonly if you are not allowed to edit it. Then we need to open it as sudo ( or root) user

The simple command to open any file using root , in graphical editor (e.g gedit) can be

# sudo gedit filename

This will open file in edit mode and as a root you can edit this. Once you are done click save and close the gedit window.

Done :)

Install SSH on Ubuntu 11.10

The default package for SSH in Ubuntu is openssh-server

To install in simple steps do this

#sudo apt-get update
#sudo apt-get install openssh-server

This would install the ssh server in your computer. If you want to change some default configuration settings in SSH then you can follow the below post. It describes the process to install SSH on Ubuntu 11.10 and how to change various settings.

http://www.jonathanmoeller.com/screed/?p=3328

Blogging client for Ubuntu

This post is part of my learnings towards movement from Windows to Ubuntu

I have been using Windows Live writer since long , so i was looking for similar kind of software for Ubuntu

After search i found the following alternatives

Blogilo
Drivel
Lekhonee

And one solution for browser based editor can be Scribefire

Similar blog posts

http://www.scaine.net/site/2010/07/blogilo/
http://askubuntu.com/questions/78593/what-blogging-clients-are-available

7 day Ubuntu Challenge for myself

I have been Windows user since pretty long , but not i want to change myself for better things.

So i have challenge for myself to use Ubuntu for one week continuously where ever i am using my own system. I would not be using Windows at all

Just trying to document my learnings as i convert myself from Windowish guy to Nix :P

I would be maintaining my daily learning log on issues which i faced and how i found answers with them.