Information Theory, Pattern Recognition, and Neural Networks





Course Videos

We recommend viewing the videos online (synchronised with snapshots and slides) at the video lectures website.

Alternatively, the videos can be downloaded using the links below. We recommend using VLC to view them.

Lecture Title Date/Time Videos Snapshots Slides Bonus Counting (labelled unrooted) trees 06 Feb 2012, 16.00 00.f4v [ 46M] 00.pdf [6.1M] 00.html Lecture 1 Introduction to Information Theory 20 Feb 2012, 16.00 01.mp4 [675M] 01.pdf [ 16M] 01.html Lecture 2 Entropy and Data Compression (I):

Introduction to Compression, Information Theory

and Entropy 27 Feb 2012, 14.30 02.mp4 [564M] 02.pdf [ 26M] 02.html Lecture 3 Entropy and Data Compression (II):

Shannon's Source Coding Theorem, The Bent Coin Lottery 05 Mar 2012, 14.30 03.mp4 [561M] 03.pdf [ 14M] 03.html Lecture 4 Entropy and Data Compression (III):

Shannon's Source Coding Theorem, Symbol Codes 16 Apr 2012, 14.30 04.mp4 [605M] 04.pdf [ 13M] 04.html Lecture 5 Entropy and Data Compression (IV):

Shannon's Source Coding Theorem, Symbol Codes and

Arithmetic Coding 23 Apr 2012, 14.30 05.mp4 [670M] 05.pdf [ 26M] 05.html Lecture 6 Noisy Channel Coding (I):

Inference and Information Measures for Noisy Channels 30 Apr 2012, 14.30 06.mp4 [588M] 06.pdf [ 22M] 06.html Lecture 7 Noisy Channel Coding (II):

The Capacity of a Noisy Channel 07 May 2012, 14.30 07.mp4 [499M] 07.pdf [ 34M] 07.html Lecture 8 Noisy Channel Coding (III):

The Noisy-Channel Coding Theorem 21 May 2012, 14.30 08.mp4 [745M] 08.pdf [ 28M] 08.html Lecture 9 A Noisy Channel Coding Gem, and

An Introduction to Bayesian Inference (I) 28 May 2012, 14.30 09.mp4 [535M] 09.pdf [ 46M] 09.html Lecture 10 An Introduction To Bayesian Inference (II):

Inference Of Parameters and Models 28 May 2012, 15.30 10.mp4 [825M] 10.pdf [ 43M] 10.html Lecture 11 Approximating Probability Distributions (I):

Clustering As An Example Inference Problem 11 Jun 2012, 14.30 11.mp4 [629M] 11.pdf [ 27M] 11.html Lecture 12 Approximating Probability Distributions (II):

Monte Carlo Methods (I):

Importance sampling, rejection sampling,

Gibbs sampling, Metropolis method 11 Jun 2012, 15.30 12.mp4 [908M] 12.pdf [ 51M] 12.html Lecture 13 Approximating Probability Distributions (III):

Monte Carlo Methods (II):

Slice sampling, Hybrid Monte Carlo,

Over-relaxation, Exact Sampling 25 Jun 2012, 14.30 13.mp4 [1.1G] 13.pdf [ 57M] 13.html Lecture 14 Approximating Probability Distributions (IV):

Variational Methods 09 Jul 2012, 14.30 14.mp4 [512M] 14.pdf [ 46M] 14.html Lecture 15 Data Modelling With Neural Networks (I):

Feedforward Networks:

The Capacity Of A Single Neuron, Learning As Inference 09 Jul 2012, 15.30 15.mp4 [950M] 15.pdf [ 92M] 15.html Lecture 16 Data Modelling With Neural Networks (II):

Content-Addressable Memories And

State-Of-The-Art Error-Correcting Codes 16 Jul 2012, 14.30 16.mp4 [1.0G] 16.pdf [ 66M] 16.html

Course page

Other course materials - free online text book [Information Theory, Inference, and Learning Algorithms] - software - further links - and errata

Our workflow, describing how the videos were recorded.