---

Search engine for webpages

---

• Inverted index
  • An inverted index is a map storing:

    keys called index terms,         values called occurence lists (= locations where the index term can be found) Wikipedia page: click here

    ---

  • Example:

  ---

  ---

  ---

  ---

• Efficient way to store the inverted index
  • Use a compressed trie:

    ---

  • Note:

    A web search engine (such as Google) does not store the webpages It searches the entire Internet and index the webpage into the inverted index

  ---

  ---

• Processing queries
  • Finding webpages containg search terms:

    For each search term, search the compress trie to find a list of webpage containing the search term. Find webpages that are listed in all outcomes Note: The values in the occurence list must be sorted by the URL to speed up this search.

  ---

  ---

  ---

  ---

• Ranking the webpage by relevance
  • All search engines try to improve the search result by "ranking" the wbepages that contains all matching search terms

  • Ranking heuristics is a research topic and it is beyond the scope of this course

  • We have in our department a specialist in webpage ranking algorithms

    I refer you to CS 572: Information Retrieval taught by Prof. Eugene Agichtein