CSE 7/5337: Information Retrieval and Web Search, Spring 2012

> Course Syllabus

Course Description

Introduces the field of information retrieval with an emphasis on its application in Web search. Introduces the basic concepts of stemming, tokenizing and inverted indices, text similarity metrics and the vector-space model. Popular Web search engines are studied and the concepts are applied in several Java-based projects using state-of-the-art frameworks like Apache Lucene.

Text

• Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, Introduction to Information Retrieval, Cambridge University Press. 2008. (Online Edition)

Slides

• Introduction and Boolean Retrieval (IIR 1)
• The term vocabulary and postings lists (IIR 2)
• Web Search (IIR 19)
• Web crawling and indexes (IIR 20)
• Dictionaries and tolerant retrieval (IIR 3)
• Index construction (IIR 4)
• Mapreduce (my installation notes for Hadoop)

Assignments

• Homework 1 due on 2/6 at noon.
• Homework 2 due on 3/5 at noon.
• Homework 3

Project

• Project 1: Web Crawling and Data Extraction due on 3/2 at noon.
• Project 2: Indexing (using MapReduce)
• Project 3: Complete Search Engine

Teams

• Eric Aceves and Lonnie Teal
• Chris Ayala
• Matthew Bolanos
• Dean Davison and Jared Otten
• Sukanya Dhamodaran
• Ethan Gill
• Kyle Hanson
• Austin Hodges
• Jaemen Moore
• Zac O'Brien and Chris White

Links

• Software
  • Install Linux (Ubuntu release 10.04 LTS) on a spare partition of your PC/Laptop or using Virtual Box.
  • Install Sun Java
  • Help on how to use a unix shell.
  • Installation of other things (netbeans, eclipse, apache, tomcat, etc.): System/Administration/Synaptics Package Manager
• Web Crawler
  • Open Source Crawlers in Java
  • crawler4j
  • Java Regular Expressions
  • Java: HTML Parser
  • Java: Working with URLs
  • About /robots.txt
• Indexing
  • Porter's Stemming Algorithm
  • Snowball: A language for stemming algorithms (demo)
  • Synonyms with Wordnet
  • MapReduce Tutorial
  • Hadoop API
• User Interface
  • Google: Basic search, operators, advanced search, (unofficial description of Google advanced search)
  • Bing: Search effectively, Bing Query language
  • Apache Tomcat
• Advanced topics
  • Apache Lucene
  • Apache Nutch
  • Apache Solr tutorial
• Interesting links
• Search engine statistics and coverage (The Search Guru)
• Google search terms

Tools for distance students

• CamStudio: Free Screen recording software for Windows.