CMSC 35900-2:
A Probabilistic Approach to Machine Learning

Fall 2008, Tuesdays and Thursdays 10:30-12:00 at TTI-C

Instructor: Nati Srebro

What is this class?

We will consider selected machine learning topics from a probabilistic, and often Bayesian, perspective. That is, we will present an approach to machine learning which focuses on constructing a probabilistic model and then predicting using the posterior probability given observations.

Pre-Requisites and Intended Audience

The class assumes an understanding of basic concepts in machine learning (not necessarily from a Bayesian perspective). It is generally not intended as an introductory class to machine learning, but rather as a class introducing an alternative perspective, and relevant techniques, to those already (at least somewhat) familiar with machine learning.


We will discuss the fundamental principles of the probabilistic approach, the relationship to other approaches, inference techniques and specific probabilistic models commonly used in machine learning.

Topics may include:


The primary text we will use for about half of the topics is:
David MacKay: Information Theory, Inference and Learning Algorithms.
The entire book, as well as some extra material, is available online (of course, you can also purchase a hardbound physical version of the book).
Other texts that will be used to cover specific topics:
Carl Rasmussen and Christopher Williams: Gaussian Processes for Machine Learning.
This book is also available online or for purchase.
Radford Neal: Probabilistic Inference Using Markov Chain Monte Carlo Methods
An excellent detailed survey of MCMC sampling techniques---available (only) online.
Andrew Gelman, John Carlin, Hal Stern and Donald Rubin: Bayesian Data Analysis, 2nd Edition.
Available for purchase, but unfortunately not available online.
Michael Jordan: An Introduction to Probabilistic Graphical Models.
This book is not yet available. Hardcopies of relevant chapters will be provided to students attending the class.

Problem Sets

Problem Set 1

Problem Set 2

Detailed Schedule and Topics Covered

Thursday October 2nd
Tuesday October 7th
Thursday October 9th: no lecture
Friday October 10th:
Tuesday October 14th: no lecture.
Thursday October 16th
Friday October 17th
Tuesday October 21st
Thursday October 23rd
Tuesday October 28th
Thursday October 30th
Tuesday November 4th
Thursday November 6th
Tuesday November 11th
Thursday November 13th
Tuesday November 18th
Thursday November 20th
Tuesday November 25th

Last modified: Tue Nov 25 22:12:43 Central Standard Time 2008