Monday, October 4, 2010

Anatomy of Search Engine.




With the rapid growth of internet , with hundreds of websites being created and millions of pages being added it was getting quite obvious that there should be some tool to store all the world’s information at one place and make it accessible to all.
The answer to this was “Search Engine”. The word Google is synonymous with search engine.
But it was in early 90’s when the search engines were introduced like Gopher and Archie. But it was in late 90’s or early 2000 Google gained into prominence by introducing a Page Rank system into its search engine. This feature was so unique from rest all other search engines (Yahoo , Alta vista and Northern Lights ) that it has made google a default website on every internet browser. Rather delving much into the history of search engines let’s look at a closer picture “ How search engines work”. Search engine feels most happy when anyone asks it to explain its own working and it will produce hundreds of results
but below is the anatomy of the search engine in the most compressed manner possible.
Search engines employ software robots called web spiders which build lists of words found on different websites. When a spider builds a list it is called web crawling. Spider first visits the most popular sites and heavily used servers. It picks the most popular words found on the page or pick the titles , headings and other special fields called meta tags.
Then the indexing is done with the word picked up by web crawler and corresponding name of the site in the database. The data base also assign a particular weight to the word based on the number of times the word appears on the page. Each information in the indexed database is encoded to save the storage space. When the user enters a keyword engine search its indexed database and give the list of best matching web pages.
Most search engine adopt the method of Page Rank to give the list of most relevant websites to the user query. Page rank also define the efficiency of search engine that whether its page ranking system is producing the most relevant results to the user query.
Revenue generation for search engines are from the advertisement posted beside the search results and each time user clicks on the ad it generates revenue for the engine or engine may charge from the company to rank their site higher.

3 comments: