How to get internet access inside Jupyter notebook

Create a file

vi ~/.ipython/profile_default/startup/

Add the following inside it
(Change proxy details as per your own environment)

import sys,os,os.path

# Test

Inside the notebook cell

import requests

Atom editor proxy

To use Atom editor behind proxy , use the below. Restart Atom if needed

apm config set strict-ssl false
apm config set https-proxy

Change proxy as per your requirements

Use Kafka command line with Kerberos

kinit -k -t myprincipal.keytab myprincipal/


/usr/hdp/current/kafka-broker/bin/ \
--zookeeper \
--topic topicname \
--from-beginning \
--security-protocol SASL_PLAINTEXT

Login to Container in Kubernetes

Step 0

Get all pods

kubectl get po --all-namespaces

Find the pod name in which your container is running

Step 1

Get the container name from running pod

kubectl describe po pod_name --namespace my_namespace

Example output

Name: pod_name
Namespace: my_namespace
Node: docker-host-03/
Start Time: Wed, 23 Nov 2016 16:57:15 +1100
Labels: name=my_container_name,pod-template-hash=2802333548
Status: Running

Note the label with value name

Step 2

Login to container


kubectl exec -it pod_name --namespace my_namespace -c my_container_name bash

Why does my LinkedIn inbox always show one unread message ?

Even if I have read all of my Linkedin messages , it still shows one unread message. Strange but true.
I am not sure what is the problem but i can tell how i fixed that.

  1. Click on any message that you have already read.
  2. Mark it as UNREAD
  3. Click on another message that you have already read.
  4. Mark it as UNREAD

See , if you notice any change in unread messages indicator ?

For me it changed correctly and then I read both messages one by one and it made my all messages as Read.

Have great time using Linkedin :)

Websites for learning Regular expressions

Regular expressions are like swiss knife which can do lot of work in small amount of code.

However knowing them deeply is very important to write code which is optimal.

I found below good websites for them while using and learning about them.

A very good website to have visual debugging for Regular expressions

Website to create and test regular expressions

A good website for Regular expressions tutorials

SBT Native Packager Building Rpm with Python files

I was working on project in which we were having Airflow python code packaged as part of RPMs. By default rpmbuild process compiles the python files which Airflow server does not likes when it seem them. Airflow likes to compile themselves. So rpm job was to ship files which raw.
Here is complete run sheet to make rpm files using Sbt native packager and disable java jar recompress and python compilation.
Edit to file project > plugins.sbt and add the plugin
addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.1.1")
Disable Java jar repackaging and Python compilation
Add the following file src/main/rpm/pre
The rpm pre scriplet is executed just before rpm is installed. Read more about rpm scripts at this link.
%global __os_install_post %(echo '%{__os_install_post}' | sed -e 's!/usr/lib[^[:space:]]*/brp-java-repack-jars[[:space:]].*$!!g' | sed -e 's!/usr/lib[^[:space:]]*/brp-python-bytecompile[[:space:]].*$!!g')

The above script is removing the bra-java-repack-jars and brb-python-bytecompile process from pre step of RPM installation.
To make the pre file work add the following in your build configuration file
import RpmConstants._ 
maintainerScripts in Rpm := maintainerScriptsAppendFromFile((maintainerScripts in Rpm).value)(
  Pre -> (sourceDirectory.value / "main" / "rpm" / "pre")

The complete rpm.sbt is shown below
import sbt.Keys._
import NativePackagerHelper._

enablePlugins(RpmPlugin, RpmDeployPlugin)

maintainer in Linux := ""
packageSummary in Linux := "Package code"
packageDescription := s"Application ${name.value}"
rpmVendor := "My company"

version in Rpm <<= version { (v: String) =>
  v.trim.replace("-SNAPSHOT", s".${System.currentTimeMillis}")

rpmLicense := Option("Apache")
rpmObsoletes := Seq(s"${name.value}")
// This does not work
rpmBrpJavaRepackJars := false
defaultLinuxInstallLocation := "/var/lib/airflow"
// This allows to override the install location using RPM prefix
rpmPrefix := Some(defaultLinuxInstallLocation.value)
import RpmConstants._

// Remove all jars
mappings in Universal := (mappings in Universal).value.filterNot{
case (file, fname) => fname.endsWith(".jar")}

// Add fat jar
mappings in Universal += {
  val fatJar = (assembly in Compile).value
  fatJar -> s"${name.value}.jar"

// Copy contents of dags folder
mappings in Universal ++= directory("src/main/dags")

// Add version.txt file
mappings in Universal += {
  val file = target.value / "version.txt"
  IO.write(file, s"${(version in Rpm).value}")
  file -> "version.txt"

import RpmConstants._
maintainerScripts in Rpm := maintainerScriptsAppendFromFile(
(maintainerScripts in Rpm).value)(
  Pre -> (sourceDirectory.value / "main" / "rpm" / "pre")

publishTo <<= version { (v: String) =>
  val artifactory = ""
  if (v.trim.endsWith("SNAPSHOT"))
    Some("Artifactory Realm" at artifactory + "artifactory/rpm-snapshots")
    Some("Artifactory Realm" at artifactory + "artifactory/rpm-releases/")

Build the rpm using sbt command
To publish the rpm use