Machine Learning for Structured Data

Course 048716 - 2009/2010
Advanced Topics in Learning, Systems and Control 2
Machine Learning for Structured Data

Instructor: Koby Crammer
Lectures: :Sunday 12:30-14:30 , Mayer Bldg, Rm. 351

Contact: 3274 or by email (find it here )
Office hours: Thursday 13:00-14:00 and by appointment

Announcements

February 21 - Please send your proposals until February 28.
February 21 - Please come and see me about exercise 1, your class presentation.
February 21 - New exercise
February 3 - The presentations will be held on Sunday, February 7, 12:00pm at Mayer 352
January 3 - New exercise
December 26 - Check project guidelines and information
December 26 - Check list of presentations
December 10 - There will be a class this Sunday, December 13.
November 22 - Check presentations guidelines and information

Course Plan

Many important problems in machine learning involve structured data: parsing sentences, model the probability of speech transcriptions, finding co-references in text. The course is an introduction to graphical models, a field which combines ideas from statistics and graph theory and attempts to provide a general representation and an algorithmic framework for reasoning about statistical dependencies. The course will introduce major tools to represent structured data, review algorithms for inference and learning, cover the main tools for theoretical analysis and demonstrate their usefulness in real world data.

Representation
- Statistical indepdendecies
- Directed and undirected graphical models
Inference
- Elimination algorithm and Junction tree algorithm
- Variational methods, belief propagation
- Sampling methods
Learning
- Parameter estimation
- Discriminative Learning, Optimization
- Incomplete data, expectation maximization (EM)
Student Presentation
- TBD

Bibliography

The first part of the course will be based on the following book. The second part will be based on resrach papers.

Daphne Koller, Nir Friedman. Probabilistic graphical models :principles and techniques, 2009. library catalog

Lectures

Lecture 1 (18/10): Introduction
Lecture 2 (25/10): Bayeisan Networks
Lecture 3 (01/11): Bayeisan Networks (cntd) + Markov Networks
Lecture 4 (08/11): Markov Networks
Lecture 5 (15/11): Elimination
Lecture 6 (22/11): Clique Trees
Lecture 7 (29/11): Message Passing
Lecture 8 (6/12): Inference using Optimization
Lecture 9 (13/12): Mean-Field + Variational Methods
Lecture 10 (20/12): MCMC
Lecture 11 (27/12): Paramter Estimation
Lecture 12 (03/01): Naive Bayes, Maximum Entropy, MEMMs
Lecture 13 (10/01): CRFs,Discrimniative Training
Lecture 14 (17/01): (Grammers)

Exercises

Exercise 1 1 (Due January 3): pdf

Grading

Two assignments: 20% (10% each)
Presentation: 20%
Project: 60%

Course 048716 - 2009/2010 Advanced Topics in Learning, Systems and Control 2 Machine Learning for Structured Data