Large scale Real time bidding for advertisement

Behavioural Targeting
Based on short term and long term activities of users
e.g User searched for new car
e.g User searched for new movie
Typical advertisement pipeline

Simple HBase data mode
Single Column family
Column Qualifier
<date><hour>:<type><value>


Challenges
User profile freshness
    Stagered freshness   
    Update after every few hours
Scaling
    Partition by geo location
    Apps interactions
Pipeline failure
HBase scaling issues
    Salting of keys for even spread
    Optimal pre spits for reading also
   
http://www.slideshare.net/Hadoop_Summit/how-did-you-know-this-ad-would-be-relevant-for-me

Now see how Yahoo solves the similar problem

http://www.slideshare.net/Hadoop_Summit/interactive-analytics-in-human-time
Yahoo makes use of druid
http://druid.io/druid.html