Syllabus

This course gives a graduate-level introduction to machine learning and statistical pattern recognition and in-depth coverage of new and advanced methods in machine learning, as well as their underlying theory. It emphasizes approaches with practical relevance and discusses a number of recent applications of machine learning, such as data mining, computer vision, robotics, text and web data processing. An open research project is a major part of the course.

In particular, the course will cover the following topics:

Supervised Batch Learning : model, decision theoretic foundation, model selection, model assessment, empirical risk minimization

: model, decision theoretic foundation, model selection, model assessment, empirical risk minimization Decision Trees : TDIDT, attribute selection, pruning and overfitting

: TDIDT, attribute selection, pruning and overfitting Statistical Learning Theory : generalization error bounds, VC dimension

: generalization error bounds, VC dimension Large-Margin Methods : linear Rules, margin, Perceptron, SVMs

: linear Rules, margin, Perceptron, SVMs Kernels : duality, non-linear rules, non-vectorial data

: duality, non-linear rules, non-vectorial data Probabilistic Models : generative vs. discriminative, maximum likelihood, Bayesian inference

: generative vs. discriminative, maximum likelihood, Bayesian inference Sequence Prediction : hidden Markov model, Viterbi

: hidden Markov model, Viterbi Structured Output Prediction : undirected graphical models, structural SVMs, conditional random fields

: undirected graphical models, structural SVMs, conditional random fields Latent Variable Models : k-means clustering, mixture of Gaussians, expectation-maximization algorithm, matrix factorization, embeddings

: k-means clustering, mixture of Gaussians, expectation-maximization algorithm, matrix factorization, embeddings Online Learning : experts, bandits, online convex optimization

: experts, bandits, online convex optimization Other topics: neural nets, ensemble methods, sparsity, ...

The prerequisites for the class are: programming skills (at the level of CS 2110) and basic knowledge of linear algebra (at the level of MATH 2940) and probability theory (at the level of MATH 4710) and multivariable calculus (at the level of MATH 1920).

Enrollment is limited to PhD students. Students who have already taken CS 4780/CS 5780 should not take this class.