• Download books
• Share and read notes on text book topics
• Watch relevant videos

Notemonk aims to create a rich and fun learning experience around text books. For launch, we’ve included the higher standards in NCERT. Please take a look and spread the work. Your feedback is most welcome!

BASH function

# wiki

# eg: wiki India
#     wiki Apple_Inc
#     wiki Anglo_Saxon
wiki()
{
    dig +short txt $1.wp.dg.cx
}

Example usage

prashanth@prashanth-desktop:~$ wiki India
"India, officially the Republic of India ( '\; see also other Indian languages), is a country in South Asia.
It is the seventh-largest country by geographical area, the second-most populous country, and the most
populous democracy in the world. Bounded by t" "he Indian Ocean on the south, the Arabian Sea on
the west, and the Bay of Bengal on the east, India has a coastline of ... http://a.vu/w:India"

prashanth@prashanth-desktop:~$ wiki Anglo_Saxon
"Anglo-Saxons (or Anglo-Saxon) is the term usually used to describe the invading tribes in the south
and east of Great Britain starting from the early 5th century AD, and their creation of the English
nation, lasting until the Norman conquest of 1066. The " "Benedictine monk, Bede, identified
them as the descendants of three Germanic tribes: http://a.vu/w:Anglo-Saxons"

Usage

prashanth@prashanth-desktop:~$ translate bonjour
hello
prashanth@prashanth-desktop:~$ translate guten morgen
Good morning

Code

#!/usr/bin/env python
'''
Translates text into english using Google Translate.
Usage: python translate.py 
        (or)
       echo  | python translate.py

For convenience, make a symlink to this file from /usr/bin/translate.
'''
# derived from : http://code.google.com/p/py-gtranslate/source/browse/trunk/gtrans.py

import sys
import urllib2
import urllib
import simplejson as json

FROM_LANGUAGE = ''
TO_LANGUAGE = 'en'
BASE_URL = 'http://ajax.googleapis.com/ajax/services/language/translate'

def translate(from_language, to_language, text):
    langpair = '%s|%s' % (from_language, to_language)
    params = {'v': '1.0', 'langpair': langpair, 'q': urllib.quote_plus(text)}

    params = '%s' % ('&'.join(['%s=%s' % (k,v) for (k,v) in params.items()]))

    url = '%s?%s' % (BASE_URL, params)
    resp = json.load(urllib2.urlopen(url))
    try:
        return resp['responseData']['translatedText']
    except:
        return text

def main(text):
    if text:
        print translate(FROM_LANGUAGE, TO_LANGUAGE, text)

    else:
        lines = [l.strip() for l in sys.stdin.readlines()]
        for line in lines:
            if line:
                text = translate(FROM_LANGUAGE, TO_LANGUAGE, line)
                print '[%s]' % line
                print text
            else:
                print

if __name__ == '__main__':
    args = sys.argv[1:]
    text = ' '.join(sys.argv[1:])
    main(text)

This algorithm has the following steps:

• make doc from html data (clean html)
• identify content nodes (nodes having substantial content)
• prune xml tree to remove irrelevant nodes
• get the most linked node from pruned tree (subtree contains relevant text)
• make the dot graph

I’ve pasted the relevant python module below for easy reading. However, if you want to download the code and hack it, you can get all the files from here.

Code files

• content_extract.py – actual work gets done here (file pasted below)
• cextract.py – cgi front-end which fetched url content and feeds to above script.
• cextract_config.py – cgi script configuration file. You have to adjust this to your environment.

Try it right here and right now

Some samples

• http://news.bbc.co.uk/sport2/hi/motorsport/formula_one/8169436.stm
• http://www.prashanthellina.com/cextract_data/640ca0b7c6e818b2b1bf952a206a6388.html
• http://www.telegraph.co.uk/news/worldnews/europe/france/5913494/Nicolas-Sarkozy-to-slow-down-after-collapsing-while-jogging.html

Please let me know if you find cases for which the algorithm does not work. Even better would be to download the code and hack it up and post back. I am eager to see what you can come up with.

#!/usr/bin/env python

import sys
from cStringIO import StringIO

from lxml import etree #http://codespeak.net/lxml/

IGNORABLE_TAGS = set(['script', 'a'])
MIN_TEXT_LEN = 50

def get_text(node):
    '''
    Given a XML node, extract all the text it contains.
    (does not recurse into children)
    '''
    text = [node.text or '']
    for cnode in node.getchildren():
        tail = cnode.tail
        if tail is not None:
            text.append(cnode.tail)

    text = '\n'.join(text).strip()
    return text

def get_xml(node):
    '''
    Convert the sub-tree from node downwards
    into string XML representation.
    '''
    return etree.tostring(node)

def create_doc(data):
    '''
    Construct XML tree datastructure from xml string representation.
    '''
    parser = etree.HTMLParser()
    doc = etree.parse(StringIO(data), parser)
    return doc

def get_content_nodes(doc):
    '''
    Identify nodes in the XML document that
    have substantial text.
    '''
    nodes = []

    for n in doc.xpath('//*'):
        tag = n.tag

        if tag.lower() in IGNORABLE_TAGS:
            continue

        text = get_text(n)
        if not text:
            continue

        if len(text) < MIN_TEXT_LEN:
            continue

        nodes.append(n)

    return nodes

def make_pruned_tree(content_nodes):
    '''
    Prune the whole XML tree by remnoving nodes
    other than content nodes and their ancestors.
    '''
    nodes = {}
    links = {}

    for node in content_nodes:

        nodes[id(node)] = node

        parent = node.getparent()
        if parent is not None:
            links[id(node)] = id(parent)

        for anode in node.iterancestors():
            _id = id(anode)
            parent = anode.getparent()
            if parent is not None:
                links[_id] = id(parent)

            if _id not in nodes:
                nodes[_id] = anode

    return nodes, links

def get_inlink_counts(links):
    '''
    Given the inter-node links, find out which
    node has maximum number of links coming into it.
    '''
    counts = {}

    for from_id, to_id in links.iteritems():
        count = counts.setdefault(to_id, 0)
        counts[to_id] = count + 1

    return counts

def get_most_linked_node(nodes, links):
    '''
    Identify the node which is most linked.
    (i,e) has most number of inlinks.
    '''
    inlink_counts = get_inlink_counts(links)

    mcount, mid = max([(count, _id) for _id, count in inlink_counts.iteritems()])
    node = nodes[mid]
    return node

def make_dot_graph(nodes, links, chosen_node, stream):
    '''
    Construct the dot format graph representation
    so that graphviz can render the tree for visualization.
    '''
    o = stream

    print >> o, "digraph G {"

    for _id, node in nodes.iteritems():

        tlen = len(get_text(node))
        tag = node.tag

        if tlen:
            text = '%s (%d)' % (tag, tlen)
        else:
            text = tag

        if _id == chosen_node:
            attrs = 'style=filled color=lightblue'
        else:
            attrs = ''

        print >> o, "%s [label=\"%s\" %s];" % (_id, text, attrs)

    for fid, tid in links.iteritems():
        print >> o, "%d -> %d;" % (fid, tid)

    print >> o, "}"

def main():
    # make doc from html data (cleans html)
    doc = create_doc(sys.stdin.read())

    # identify content nodes
    content_nodes = get_content_nodes(doc)

    # prune xml tree to remove irrelevant nodes
    nodes, links = make_pruned_tree(content_nodes)

    # get the most linked node from pruned tree
    mnode = get_most_linked_node(nodes, links)

    # make the dot graph
    make_dot_graph(nodes, links, id(mnode), sys.stdout)

if __name__ == '__main__':
    #Eg: wget "http://blog.prashanthellina.com" -O - | python thisscript.py | dot -Tpng -o /tmp/test.png ; eog /tmp/test.png
    main()

Features like “Places” (Bookmarks on steroids), Cairo for rendering and OS specific widgets have made the best browser better. At the time of writing of this blog post 6 million plus downloads from around the world have already happened.

Everywhere I look at work, I see the “Download Day” certificate from Mozilla corp … I got one myself too

But the question is ….

Do you know who else got the certificate ???

Who else but Billy and Ballmy

I got inspired to bring my long standing idea to fruition and the outcome is Wordza.

Wordza is a smart word quizzing service designed to help you rapidly learn new words. It adapts to your level and continually challenges you to get better. Preparing for GRE/SAT/CAT? Use Wordza 15 minutes a day.

Questions have difficulty ratings ranging between 1 and 100. The ratings are derived from user activity. A question is easy if a lot of people get it right and difficult when very few do!

You start at difficulty level 50. If you get three questions right in a row, you drive up the difficulty level a notch. On the other hand, answering a question wrong will reduce the difficulty level by one.

If you are interested in the implementation … I used Python, MySQL behind Apache to get this up. When I started making it, I wanted to keep it simple and make it work fast. I hope I’ve met both these requirements. Do give it a try and let me know what you think.

Alexa collects statistics about visits by internet users to websites through the Alexa Toolbar. Based on the collected data, Alexa computes site ranking. By examining the Alexa rank of a site, you can get a rough idea of how popular the site is. Many argue that Alexa rank is misleading but it has its uses.

The Alexa rank script

You can find out the Alexa rank for any site by using this page. However, if you want to programatically get the Alexa rank, you can do it using this script.

Get the Alexa rank script

Using the script

After downloading the script, give it execute permission by doing this. You will need to have Python installed.

chmod +x get_alexa_rank.py

$ ./get_alexa_rank.py google.com
popularity rank = 2
reach_rank = 1

$ ./get_alexa_rank.py wikipedia.com
popularity rank = 7
reach_rank = 6

$ ./get_alexa_rank.py blog.prashanthellina.com
popularity rank = 557287
reach_rank = 482289

$ ./get_alexa_rank.py www.inexistantsite.com
popularity rank = -1
reach_rank = -1

How does the script work?

If you make a http request for the following url,

http://data.alexa.com/data?cli=10&dat=snbamz&url=$URL

after replacing $URL with the url of the site for which you need the Alexa rank, the following XML response is sent out. I tried with “http://blog.prashanthellina.com”.

The script parses the XML response and extracts POPULARITY/@TEXT and REACH/@RANK.

If you are looking for a PHP script for doing the same, check this out.

Let us name a domain!

http://www.domaintools.com/

DomainTools takes a concept as input and comes up with domain name suggestions. Let us say you are starting a website about “Vacations in Mexico”. Go to their website and type in “Mexico Vacations” in the text box and click on the button to get suggestions.

Domain name suggestions for “Mexico Vacations”

MexicoFreeTours.com
GreatMexicoTrips.com
MexicoGreatBreaks.com
MyMexTrips.com
FunMexVacations.com

Not bad at all. I like GreatMexicoTrips.com.

Another tool out there is Noemata Domain name generator. This one gives random domain name suggestions. Not very impressive.

kawlant.com
phobolapa.com
jupidox.com
nitrurth.com

Doing a web2.0 start-up? Don’t sweat over the name

Flickr, tumblr, zoho – all taken up? Find more!

The Web2.0 name generator

Gedel
Eavu
Plamba
Trilith
Zooveo

Another Web2.0 name generator

65pages
inclicks
type17
bubbledesks

Another one! – This one checks domain availability too. sweet.

Skiplounge
Mivee

Web2.0 buzz generator

2.0 is the new New. It’s all changing. Label what defies categorization. “ASL” is geezer speak. Podcasts. The words aren’t what they were. Clear that. Hack it. Faster. Faster! Float this. Roll your own roll-your-own. MSM just doesn’t get it. Always be launching. The buzz is loud and clear. You need someone who gets it. On-demand streams. Splog is an aggregate noun. We are on the brink of a new age. This will change everything. We’re about what Web 2.0 is about. Single. Word. Sentences! News clouds. An AJAX-driven GUI. Cry out, blogosphere! This is newer media. The new is old. Tag me. We shall transcend borders. Clustering. It’s all about community. Social is the new push. Folksonomy.

Starting a band? Check out the band name generator

B.A.N.G – Band automatic name generator!
Subtle Alphabet
Uncommon Elasticity
Notorious Sand
Crass Ink

Play Civ3? Want a name for your new city?

The place name generator

Agea
Byzernova
Genesini
Mercentia
Myruaror

The fantasy place name generator

Aldmeadow
Blacklake
Southshore
Wildecoast
Witchmead

The British town name generator

Cuddleballabridge, Cambridgeshire
Buttercanter Gate up the Snotty, Wiltshire
Old Chippingbeck, Shropshire
Dorkstrathsack in the Gloaming, Somerset

Game name generators

RPG Name Generator

Gilder’s Airship
Ruby Law V
Sad Guardian
Star: Ghost of the Interstellar Air
Unnamed Peace XI

The Ultimate game name generator

Aqua Badger
Nuclear Dazzler

Castlevania – game name generator

Surf Saxophone of Dog Poo
Minuet of Skunky Beer
Bathroom Break of Stubbed Toes

The Video game name generator

Intelligent Kangaroo Park
Spectacular Dentist All-Stars
Master Chief’s Fantasy Rocket

Character name generator

Ageroth
Cheon
Hagiti
Roddareld

Name your dream gadget!

Wacky gadget name generator

Atomic Chronostinker
Computerized Golfing Zapperion
Configurable Barfer
Configurable Electromagnetic Lithowhiner
Dimension Whiner

Name a corporation or government

Corporation name generator

Boyd Science Logistics Holdings
Cosmotechnics
Estrada Medical Transportation
Genetics Cosmosystems
Gold Informational Solutions

Government generator

Ancient Imperial Unity
Constitutional Brotherhood
Dominion of States
Provincial Authority of Territories

Name the pub

Pub name generator

The tenrec & indian
The boa constrictor & bar
The kudu & deal
The serval & guide

Cook up a research topic for your thesis !

A research topic generator!

A virtual high-speed agent embedded in a collaborative digital interface
An active mobile display embedded in a meta-level programmable solution
A high-level logical preprocessor related to an interactive concurrent theorem prover
A responsive high-speed preprocessor applied to an active proxy network

Throw some insults around!

An insult generator

Sneezefinger
Muckfoot
Bumface
Wadface

People names!

Greek names

Turatia
Dodius
Struinaus
Kymius

Japanese names

Omira
Ayuro
Ekochiyu
Hetamuri

Cook up some colors

Color name generator

Cobalt Mesa
French Latte
Slate
Southern Maroon
Swirl Purple

I am sure there are many more around the web that I missed. If you find anything interesting, comment about it!

Every once in a while, someone comes up with a way of doing things in an extremely obvious and simple way. When this happens, a zillion others say, “of course that’s the way to do it!”. Songza is a music search engine and jukebox that is dead simple to use. You should try it to really grok how simple the interface is.

Songza is the single best place to search for and listen to music on the Web. Find a song and listen immediately, in one place.

Unlike KaZaa or Bit Torrent, Songza users can only listen to songs, not download them. And unlike Last.fm or Rhapsody, Songza permits users to choose exactly the song or artist they want to hear, and does not require them to subscribe or pay for its services.

Songza is the brainchild of 24-year-old Aza Raskin — the son of Apple Macintosh founder Jef Raskin. He and Scott Robbin worked feverishly to bring the idea to life. The site launched on November 8, 2007, and instantly generated significant buzz around the world. Hundreds of bloggers and news writers have praised its elegant user interface, beautiful design, and all-around utility. Like all good ideas, both the concept and design came to Aza while he was in the shower.

The thing I liked most about songza is the “pie-menu”.

useful links

• http://ajaxian.com/archives/songza-would-you-like-a-habituatable-pie-menus-with-your-social-music

• http://www.downloadsquad.com/2008/03/04/songza-adds-new-features/
• http://humanized.com/weblog/2007/11/13/songza-launch/
• http://blog.wired.com/music/2007/11/songza-turns-yo.html
• http://lifehacker.com/software/music/listen-to-music-on+demand-with-songza-321638.php

Along with storing the books, I generated thumbnails for every chapter. Navigation pages have also been generated to help browse through the books easily. You can access this dump here.

The crawler code is here.

These are the steps to run it yourself.

# scrape ncert site to get details about book pdf urls
python get_ncert_books_data.py > ncert_books_data

# create a directory to store all downloaded files
mkdir ncert_books

# download all the pdfs and cover page images
cat ncert_books_data | python download_ncert_books.py ncert_books

# generate thumbnails for each pdf (requires ImageMagick)
find ncert_books/ -iname "*.pdf" > all_pdf_files
sed -i "s/^/0\t/" all_pdf_files
python generate_thumbnails.py

# resize the book cover images
python resize_image_thumbnails.py ncert_books

# generate the navigation pages
python generate_navigation_pages.py ncert_books

For the thumbnail generation to work, you will need ImageMagick. Read this for more information.

Google has announced “Knol”. In their words

a new, free tool that we are calling “knol”, which stands for a unit of knowledge. Our goal is to encourage people who know a particular subject to write an authoritative article about it.

…
The key idea behind the knol project is to highlight authors.
…
We believe that knowing who wrote what will significantly help users make better use of web content. At the heart, a knol is just a web page; we use the word “knol” as the name of the project and as an instance of an article interchangeably. It is well-organized, nicely presented, and has a distinct look and feel, but it is still just a web page.
…
Google will not serve as an editor in any way, and will not bless any content. All editorial responsibilities and control will rest with the authors.
…
At the discretion of the author, a knol may include ads. If an author chooses to include ads, Google will provide the author with substantial revenue share from the proceeds of those ads.

I am not impressed by what I hear. The model might encourage editors to start gearing their articles to get more Ad hits and the quality and correctness of information may take a downward turn.

However I do see some sense in one person controlling an article and accepting “patches” from others and incorporating them. This way maintenance of an article becomes less tedious than a comparable Wikipedia article. The model is somewhat like an open source project. You can contribute patches to the maintainers and they will be included after screening. If you don’t like the way the project is being handled, you can always fork it and maintain your own.

I am going to wait and watch to see what happens. Will Knol go the Orkut way or not?

Links

• http://googleblog.blogspot.com/2007/12/encouraging-people-to-contribute.html
• http://en.wikipedia.org/wiki/Knol
• http://blog.sairahul.com/2007/12/google-knol-it-is-quite-surprise-news.html
• http://www.techcrunch.com/2007/12/14/google-knol-a-step-too-far/
• http://www.metafilter.com/67449/Googles-Knol

• 220 GB disk space and grows by 2GB every week
• 2TB bandwidth per month and grows by 16GB every week
• SSH access
• Build environment, gcc
• Python, Perl, PHP
• Unlimited mysql databases and tables
• Free domain registration
• Can host unlimited domains and sub-domains
• Crontab
• Subversion!
• One click installs (I installed WordPress this way)

These were the features when I got my account. However DreamHost is celebrating their 10th anniversary and starting disk space is 500GB and starting bandwidth per month is 5TB! For all this you pay $9.95 per month!

I was very wary as all this seemed too good to be true. After some googling I found that a lot of people liked DreamHost and the support is excellent (both from the community and the company itself).

DreamHost did have growing pains and had a major outage last year and I did notice that my site was inaccessible a couple of times for a duration of 2-5 minutes. Also the load average on the machine I was hosted, touareg, was 35 – 40. However, things have been improving. The load average is down to 10 and I have seen that during extended periods of time it even drops down to the 2 – 3 range. The server are pretty good too. AMD Opterons with 4GB of RAM.

You can even do some minimal back-end processing without too much hassle. I’ve noticed that processes which take less than around 88MB are allowed to stick around while those over are killed promptly.

If you are looking for a good host with tons of features, you need look no further. You can thank me for good advice by specifying me as the referrer when you sign-up. DreamHost has a referral program and I just might get referred enough times to cover my hosting fee! This works for you too after you register.

Get an account with DreamHost here

And don’t forget to use this promo code which will save $80 when you sign up.
Promo code : PRASH80

Here is the embed code. Enjoy I will blog shortly about other interesting ways of embedding this.