Hive use Derby database by default for storing its data.
But it has limitation that only one user can access it and the data cannot be shared among multiple machines.
So we can use MySQL Database to store the metadata in hive.
Go to hive-site.xml
and configure the following properties
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://hostname:3306/hive?createDatabaseIfNotExist=true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hadoop</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hadoop</value>
</property>
Create one use hadoop in MySQL and give its password chosen above with grant all privileges
CREATE USER 'hadoop'@'hostname' IDENTIFIED BY 'hadoop';
GRANT ALL PRIVILEGES ON *.* TO 'hadoop' WITH GRANT OPTION;
Thats it :)
Start using Hive
But it has limitation that only one user can access it and the data cannot be shared among multiple machines.
So we can use MySQL Database to store the metadata in hive.
Go to hive-site.xml
and configure the following properties
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://hostname:3306/hive?createDatabaseIfNotExist=true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hadoop</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hadoop</value>
</property>
Create one use hadoop in MySQL and give its password chosen above with grant all privileges
CREATE USER 'hadoop'@'hostname' IDENTIFIED BY 'hadoop';
GRANT ALL PRIVILEGES ON *.* TO 'hadoop' WITH GRANT OPTION;
Thats it :)
Start using Hive
Hi
ReplyDeleteI'm trying to setup hive on single node. I would really appreciate your help if you could please point me where I'm going wrong.
I setup hadoop on Ubuntu 10.04 single node by following the steps mention in:
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/#sun-java-6
I believe that Hadoop is setup and configured properly. Then, I downloaded stable release of hive from apache hive as .tar.gz file and decompressed it and moved to the desired location. Also, I set the path variables but when I try to test hive by running any query I get a huge error statement ending with:
Cleanup action completed
FAILED: Error in metadata: javax.jdo.JDOFatalDataStoreException: Failed to create database 'metastore_db', see the next exception for details.
NestedThrowables:
java.sql.SQLException: Failed to create database 'metastore_db', see the next exception for details.
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
My Hadoop directory: /usr/local/hadoop/
My Hive directory: /usr/local/hive/
Path variables I set:
export PATH=$PATH:/usr/local/hive/bin/
export PATH=$PATH:/usr/local/hive/lib/
export PATH=$PATH:/usr/local/hadoop/bin
The hive is not able to create metastore_db file inside directory where you started hive. Do you have write permissions for that directory ?
DeleteI have done the mysql and hive configuration by referring to your blog.But when i am running mysql with user hadoop,there is no hive database created inside the mysql.
DeleteHi,
DeleteDo you have
createDatabaseIfNotExist = true?
Have you given permissions for the user hadoop
CREATE USER 'hadoop'@'hostname' IDENTIFIED BY 'hadoop';
GRANT ALL PRIVILEGES ON *.* TO 'hadoop' WITH GRANT OPTION;
Can you check with root user once if its created or not
yes i checked that also and in my hive-0.10.0/conf folder there is no hive-site.xml file,so i do made changes in hive-default.xml.template.hive database is not created in root but when i created one database hive and use that hive,there was no tables inside hive.
DeleteBy default, Hive uses a built-in Derby SQL server, which provides limited, singleprocess storage. For example, when using Derby, you can’t run two simultaneous instances of the Hive CLI
ReplyDeleteso check if there are 2 Hive CLI running
This comment has been removed by the author.
ReplyDeleteThis error can be due to incompatibility between version of hadoop and hive supported
DeletePost your Hive and Hadoop version
This comment has been removed by the author.
DeleteThis comment has been removed by the author.
ReplyDeleteHadoop 0.22.0 and hive 0.9.0.But i tried hive 0.8.0 also but it didnt work.Same error is coming everytime
ReplyDeleteThere is no hive-site.xml present in hive-0.9.0-bin/conf folder.So what to do Sir?
ReplyDeleteHi apology for delay,
DeleteJust create one xml file and put the content mentioned above.
Also the property to tell where is your job tracker
mapred.job.tracker
JobTracerIP