Build and compile Hadoop from source

Install some base libraries

apt-get -y install maven build-essential protobuf-compiler autoconf automake 
libtool cmake zlib1g-dev pkg-config libssl-dev


Checkout code from repo

 svn co http://svn.apache.org/repos/asf/hadoop/common/trunk/ hadoop
cd hadoop/

mvn compile


This gave me

[ERROR] Could not find goal 'protoc' in plugin org.apache.hadoop:hadoop-maven-plugins:3.0.0-SNAPSHOT among available goals -> [Help 1]

https://issues.apache.org/jira/browse/HADOOP-9383

Then to debug i re run with

mvn compile -X

I found in log that i did not had version 2.5 of protobuf required to build hadoop

Follow the steps here to install protobuf on your system

http://jugnu-life.blogspot.com/2013/09/install-protobuf-25-on-ubuntu.html

Check the installed version of protobuf , its also documented in

https://svn.apache.org/repos/asf/hadoop/common/trunk/BUILDING.txt

Okay after fixing above ( if required )

Here is final build command

mvn package -Pdist -DskipTests -Dtar

Further reading


http://wiki.apache.org/hadoop/HowToContribute



Install protobuf 2.5 on Ubuntu

Check which version you have

You can use synaptic package manager for finding it

If you want to install manually

Copy paste in some script below commands

sudo apt-get install build-essential
mkdir /tmp/protobuf_install
cd /tmp/protobuf_install
wget https://github.com/protocolbuffers/protobuf/releases/download/v2.5.0/protobuf-2.5.0.tar.gz
tar xzvf protobuf-2.5.0.tar.gz
cd  protobuf-2.5.0
./configure
make
make check
sudo make install
sudo ldconfig
protoc --version 


Check the installed version

$ protoc --version
libprotoc 2.5.0



Why we need ldconfig read link below
http://code.google.com/p/protobuf/issues/detail?id=213 

Related error searches

[DEBUG] protoc: error while loading shared libraries: libprotoc.so.8:
protoc: error while loading shared libraries: 
 
[DEBUG]  libprotobuf.so.6: cannot open shared 
object file: No such file or directory
 
Cannot find `protoc` command 
 
If you came to this page with above error search then please do
 
sudo ldconfig
 
The details are present on link above
 
 

Change Eclipse java and memory settings

Eclipse can be told to use specific version of Java if you have multiple on your system

Open eclipse.ini file

Add the following in it , see the entry i made in bold

org.eclipse.platform
--launcher.XXMaxPermSize
2560m
--launcher.defaultAction
openFile
-vm
/home/jagat/development/tools/jdk1.6.0_45/bin/java

--launcher.appendVmargs
-vmargs
-Dosgi.requiredJavaVersion=1.6
-XX:MaxPermSize=2560m
-Xms400m
-Xmx5120m

Change memory settings

You can increase the momory allocated for eclipse here.

If you notice carefully , in all of the above i added one extra 0 , e.g 40 i made 400

You can change depending on your system need.

Useful websites for maven

Some of the websites which you can use while working with maven

http://mvnrepository.com

http://search.maven.org




Eclipse menu not working ubuntu 13.10

I downloaded eclipse on Ubuntu 13.10 and when i open eclipse you know that for all applications ubuntu show menu at top bar. Eclipse was also not exception.

However strange thing was i could not do anything with those links.

So i searched and found this bug

https://bugs.launchpad.net/ubuntu/+source/indicator-appmenu/+bug/659931

Read if you like

How to solve

Instead of clicking eclipse directly start it with following command

 UBUNTU_MENUPROXY= /home/jagat/development/tools/eclipse/eclipse

Where the path on right side is exact path where your eclipse is sitting.

This should bring menus back in eclipse like normal OS and it would work.

However i am still thinking why this did not work in Ubuntu




Ubuntu safe remove usb drive

To safely remove the usb disk drive from Ubuntu

Right click on USB Disk drive

Select

Safely remove parent drive


Getting started with Scala IDE Tutorial 1

 

Download the Scala editor

http://scala-ide.org/

Create new Scala Project

image

Create first Object

File > New > Scala Object

image

Write first Class

image

Run As Scala Application

Control + F11

See the output in Console

Congrats : You have run your first Scala IDE program

Lets see next