Hadoop Windows Cygwin error tmp folder file permissions

Error
java.io.IOException: Failed to set permissions of path: file:/tmp/hadoop-jj/mapred/staging/jj-1931875024/.staging to 0700
The details for my Hadoop were 0.20.203
Solution
This is known issue in 203 version , just download the previous stable version from hadoop website
OR
Download the latest stable version from Cloudera website , incase you are using that flavour
Details
Also you can read the details for possible causes for the above error at below link
http://lucene.472066.n3.nabble.com/Problem-debugging-MapReduce-job-under-Windows-td2913135.html

 You can also develop using eclipse in Windows , for more details on how to setup eclipse development environment for hadoop please follow the link http://jugnu-life.blogspot.com/2011/11/hadoop-development-environment-in.html

Hadoop Setup in Windows

The below post will explain quickly how to install hadoop on windows for development

I used Windows XP  , you can follow the steps to configure standalone setup for hadoop

 

  1. Download the latest stable release from hadoop website
  2. If you are on Windows you also need Cgywin , download from its website and choose to install openssh and rsync packages
    (When you download hadoop in the file /src/core/overview.html , You can find the list of softwares required )
  3. Add path varible in windows as ;c:\cygwin\bin;c:\cygwin\usr\bin
  4. Copy hadoop zip file to /usr/local directory , rename the hadoop.x.x folder to simple name such as hadoop
  5. To make the JAVA_HOME setting up easy , i copied java directly into usr\local\lib\jdk1.7.0 folder
  6. Open the file called conf/hadoop-env.sh and go to JAVA_HOME variable
    Set something like
    # The java implementation to use.  Required.
    export JAVA_HOME=/usr/local/lib/jdk1.7.0
    If you are copying the JAVA in default windows path like C:\Program Files\Java then You need to escale space with \
  7. To test that configuration has been done properly do this on cgywin command prompt

$> cd /usr/local/hadoop
$> bin/hadoop version

It would show output like

Hadoop 0.20.203.0
Subversion http://svn.apache.org/....
Compiled by XXX on XXX

Congrats :) Its done

Junit Tutorial

This is one of the post  in series of simple Junit tutorials. I loved the book Junit in Action so lots of content is inspired by that book.

I will be explaining with simple example and also from time to time pointing you to read certain pages from API of Junit , so that you can have solid foundation of junit.

Junit is used for unit testing of code we write. When we say unit testing we mean testing as simple as one simple method to see if its is playing as per our expectations.

I will write one simple Calculator Class which has one simple method to add two numbers. We will write junit test to see if our method is implemented in correct way or not.Although this can seem to you , its foolish to test the addition of two numbers , but motive here is to understand the working of junit.

Before going through the tutorials you would like to setup your development environment for the tutorials. Read the previous post on setting up junit while using Ant task



package com.learnjunit;
public class Calculator {

public double add(double number1, double number2) {
return number1 + number2;
}
}

We have one simple Calculator class and one method add which is taking two numbers and returning their sum


Now we would write test case for this class


Best Practice : Name the test class as class with same name as class being tested + word Test so the test class in our case would CalculatorTest.java




package com.learnjunit.test;

import com.learnjunit.*;
import org.junit.Test;
import static org.junit.Assert.*;

public class CalculatorTest {

/**
* Test annotation tells that this method can be tested by junit It can also
* accept two things
*
* @Test(expected=IndexOutOfBoundsException.class)
* @Test(timeout=100)
*/

@Test
public void testadd() {
Calculator c = new Calculator();
//Asserts that two doubles or floats are equal to within a positive delta.
double result=c.add(10,20);
// Check if result is equal to expected value of 30
// IF they are not equal , exception is throws with given message
assertEquals("Checking add method failed",30,result, 0);

}
}


 

 
 
 

The line by line explanation for the CalculatorTest.java is as follows


Line 1


We created this class in separate package com.learnjuit.test , its always good practice to keep test code separate from code being tested


Line 3


We are importing the package of class which is being tested ( in this case Calculator.java )


Line 17


We created one method named testadd() which is having annotation @Test , this annotation tells that this method has to be taken up by junit


We created one Calculator object then calculated result based on implementation of add method in Calculator.java


We used assertEquals method to verify if the result is same as the expected one.


 


Things to read from Junit API


Test


Assert ( See various assert Methods)


If you have read the previous tutorial on setting up Ant Junit task then you can run it from command prompt


Go to folder where the build.xml file is


ant junit


The complete test report would be published in the target/test-report folder


You can download the complete source code from this link

Running Junit test cases using Ant

This tutorial explains setting up a simple junit example using Ant Junit task

Ant has task named Junit which can be used for unit testing using Junit 3.0 and Junit 4.0 framework. To run junit using Ant you need junit-4.9.jar ( or latest ) and ant-junit.jar .

ant-junit jar would be already there in ant lib folder so you don’t have to do anything for that. Incase you need help in setting up Ant for this tutorial , just read the tutorial on how to setup ant for your system.

You can download latest junit-x.y.jar from junit website

To do simple unit testing i have created one simple Calculator class and its corresponding test class.

No need to copy manually , You can also download the full code for this tutorial from link mentioned below

Code for Calculator.java

package com.learnjunit;
public class Calculator {

public double add(double number1, double number2) {
return number1 + number2;
}
}

And its corresponding test class is CalculatorTest.java

package com.learnjunit.test;

import com.learnjunit.*;
import org.junit.Test;
import static org.junit.Assert.*;

public class CalculatorTest {

@Test
public void testadd() {
Calculator c = new Calculator();
//Asserts that two doubles or floats are equal to within a positive delta.
double result=c.add(10,20);
// Check if result is equal to expected value of 30
// IF they are not equal , exception is throws with given message
assertEquals("Checking add method failed",30,result, 0);

}
}

Now comes the main part , the build file for junit task

  1: <project name="learnjunit" default="junit">
  2: <description>Build file to do junit testing</description>
  3: 
  4: 	<property file="build.properties" />
  5: 	<property name="src.dir" location="src" />
  6: 	<property name="test.dir" location="test" />
  7: 	<property name="src.dir.classes" location="${src.dir}/classes" />
  8: 	<property name="test.dir.classes" location="${test.dir}/classes" />
  9: 	<property name="junit.lib.dir" location="lib" />
 10: 	<property name="test.result.dir" location="target" />
 11: 
 12: 	<path id="junit.lib">
 13: 		<fileset dir="${junit.lib.dir}">
 14: 			<include name="**/*.jar" />
 15: 		</fileset>
 16: 		<pathelement location="${src.dir.classes}" />
 17: 	</path>
 18: 
 19: 	<target name="clean" description="Delete all directories which are created as part of build">
 20: 		<delete dir="${src.dir.classes}" />
 21: 		<delete dir="${test.dir.classes}" />
 22: 		<delete dir="${test.result.dir}" />
 23: 	</target>
 24: 
 25: 	<target name="init" depends="clean" description="Create classes directory for compilation">
 26: 		<mkdir dir="${src.dir.classes}" />
 27: 		<mkdir dir="${test.dir.classes}" />
 28: 	</target>
 29: 
 30: 	<target name="compile.src" depends="init" description="Compile to source code">
 31: 		<javac srcdir="${src.dir}" destdir="${src.dir.classes}" />
 32: 	</target>
 33: 
 34: 	<target name="compile.test" depends="compile.src" description="Compile Junit test classes">
 35: 		<javac srcdir="${test.dir}" destdir="${test.dir.classes}">
 36: 			<classpath>
 37: 				<pathelement location="${src.dir.classes}" />
 38: 				<pathelement path="${junit.lib}" />
 39: 			</classpath>
 40: 		</javac>
 41: 	</target>
 42: 
 43: 	<target name="junit" depends="compile.test" description="Run the junit test">
 44: 		<mkdir dir="${test.result.dir}/test-results" />
 45: 		<junit fork="true" forkmode="perBatch" haltonfailure="false"
 46: 			printsummary="true" dir="${test.result.dir}" failureproperty="test.failed">
 47: 			<classpath>
 48: 				<pathelement location="${src.dir.classes}" />
 49: 				<pathelement location="${test.dir.classes}" />
 50: 			</classpath>
 51: 			<formatter type="brief" usefile="false" />
 52: 			<formatter type="xml" />
 53: 			<batchtest todir="${test.result.dir}/test-results">
 54: 				<fileset dir="${test.dir.classes}">
 55: 					<include name="**/*Test.class" />
 56: 				</fileset>
 57: 			</batchtest>
 58: 		</junit>
 59: 		<mkdir dir="${test.result.dir}/test-report" />
 60: 		<junitreport todir="${test.result.dir}/test-report">
 61: 			<fileset dir="${test.result.dir}/test-results">
 62: 				<include name="TEST-*.xml" />
 63: 			</fileset>
 64: 			<report format="frames" todir="${test.result.dir}/test-report" />
 65: 		</junitreport>
 66: 		<fail if="test.failed" />
 67: 	</target>
 68: 
 69: </project>

In junit target , it first created one directory where it can place the test results


In classpath for junit we are giving the classes folder for both source and test java files. These class files are created by targets compile.src and compile.test


When junit target runs it see all the files in test.dir.classes which end with .class and consider them for batch run as junit.


In the junitreport we are making report in the form of frames similar to javadoc reports for hava source code.


 


To run this example , download complete source zip and extract it to your computer.


Open command prompt


run command like


ant junit


The build successful message will appear


The build will create target directory and place the junit test results


Thanks for reading , please share your comments

CSS3 Speech Module

CSS3 Speech module is now in last call state (Aug 18 , 2011) . Journey for making accessible web standards have come far from WCAG guidelines , SSML (Speech Synthesis Markup Language ) to CSS Speech

Here is the summary of what CSS Speech module provides.

CSS Speech standard specifies how to present the content in way which can be understood by speech synthesizers. Using CSS Speech module we can specify meta data like voice speed , pause information etc for content . This CSS information is used by User agents while rendering and producing final sound output for the content.

Typical usage for this would be

  • Screen readers
  • Machine to Human interaction ( e.g incase while you are driving car , car would read content for you online with appropriate pauses , breaks etc or your ebook reader will read the book , exactly the way it is read by humans)

CSS Speech is based on concept of Aural Box Model for content to be rendered in speech form . It is a conterpart for visual box model for visual presentation of content.

See the Visual box model for CSS and Aural Box Model below for similarities

CSS Aural Speech Model and CSS Box Model

 

Cue : Cue is small piece of sound which you can insert in between.

Properties and Characteristics which are available include

Pause Properties

  • pause-before
  • pause-after

Rest Properties

  • rest-before
  • rest-after

Cue Properties

  • cue-before
  • cue-after

Speaking Properties

  • speak
  • speak-as

Mixing Properties

  • voice-volume
  • voice-balance

Voice Characteristics

  • voice-family
  • voice-rate
  • voice-pitch
  • voice-range
  • voice-stress

The link below provides the complete summary for each of the above properties

http://www.w3.org/TR/css3-speech/#property-index

Browser share in various markets

Browser share worldwide shows some interesting ideas.

Internet Explorer will slowly fade down to market share of 20%

You can develop with latest standards of CSS3 and HTML5 freely

Ultimately browsers will compete with them selves to retain market share , following latest W3C specifications for CSS and HTML

As a Web Designers / Developers we can recommend our users to use latest browsers which can give best browsing experience

 

Here are some of the useful website to check the market share statistics for latest browsers

 

 

My main point for looking at these websites was something different , i wanted to analyze how various counties fair in terms of riding the latest browsers usage wave. Which country is ahead in using up latest browsers.

Graceful degradation and Progressive Enhancement

Graceful degradation and Progressive enhancement are two different web design strategies

Progressive enhancement

In progressive enhancement you design for bare minimum design standards , generally without any CSS or JavaScript support. In this way webpage will have content that all users can see at least , irrespective of the user agents.

And slowly slowly on the top of baseline design we add on latest standards support for example Latest CSS3 styling.

You can read more about Progressive enhancement at Wikipedia

Graceful degradation

Graceful degradation is very aggressive form of web design in which you are following latest standards and specifications and incase user agent doesn't support them , you don't show latest markup styles. Your webpage just degrades itself to present all what is supported by current user agent.

You can read more about Graceful degradation at Wikipedia

Some of the useful links on the same topic

Also please share any other useful link you have for the same.