INF8953CE: Machine Learning (Fall 2020)

Tentative Schedule for Fall 2020

Lecture videos are available here.

Lecture scribbles are available here.

Lec.
Date
Topic
Lecture Material
Reference
Readings
1
Aug. 31

Introduction to machine learning.
Slides, chapter-01 Bishop section 1.1 Computing Machinery and Intelligence
Probability Review-1
Probability Review-2
Linear Algebra Review
2
Sep. 03

Regression and Overfitting.
chapter-01 Bishop section 1.1,
HTF section 2.3
Sep. 07

HOLIDAY.
Singular Value Decomposition.
3
Sep. 10

Overfitting, ML Pipeline, Classification.
chapter-01 Bishop section 1.1,
HTF section 2.3
4
Sep. 14

Statistical Decision Theory, More on Linear Regression.
Chapter-02, Chapter-03 HTF section 2.4, 2.5
Bishop section 3.1
Linear Algebra review (7.1) in this pdf
Lagrange Multiplier (Appendix E in Bishop)
5
Sep. 17

Gradient Descent, Regularization.
Chapter-03 Bishop section 3.1, 3.1.4
HTF section 3.4.1, 3.4.2
6
Sep. 21

Bias-variance Tradeoff, Linear Classification.
Chapter-04, Chapter-05 Bishop section 1.5.5, 3.2, 1.5.4, 4.1.1, 4.1.2 Intro to convex optimization (page 91 to 102 in this book)
7
Sep. 24

Linear Classification.
Chapter-05 Bishop section 1.5.4, 4.1.1, 4.1.2, 4.1.3
8
Sep. 28

Dimensionality reduction, PCA, LDA.
Chapter-06 Bishop section 4.1.4
this pdf for pca.
No new reading. Finish previous reading.
9
Oct. 01

GLM, GDA.
Chapter-07 Bishop section 4.2
10
Oct. 05

GDA, Naive Bayes.
Chapter-07 Bishop section 4.2 Bishop section 2.3
11
Oct. 08

Evaluation metric for classification, Logistic Regression.
Chapter-07, Chapter-08 Bishop section 4.3
Oct. 12

Study Break.
Oct. 15

Study Break.
12
Oct. 19

Logistic Regression, Newton-Raphson Method, Perceptron.
Chapter-08, Chapter-09 Bishop section 4.3, 3.1.1, 1.5.4, 4.1.7.
HTF section 4.5.
Generative and discriminative classifiers by Tom Mitchell.
13
Oct. 22

Perceptron, Max-margin Classifier.
Chapter-09 Bishop section 4.1.7, 7.1
HTF section 4.5
14
Oct. 26

Max-margin Classifier, SVM.
Chapter-09 Bishop section 7.1 A Few Useful Things to Know about Machine Learning by Pedro Domingos
15
Oct. 29

Non-parametric methods, Decision Trees.
Chapter-09, Chapter-10 Bishop section 7.1
TSKK section 3.3
Mitchell Chapter 3
16
Nov. 02

Decision Trees, Bagging, Random Forests, ERTs.
Chapter-10, Chapter-11 TSKK section 3.3
Mitchell Chapter 3
Bishop Chapter 14
No new reading. Finish previous reading.
17
Nov. 05

Boosting.
Chapter-11 Bishop Chapter 14
18
Nov. 09

Stacking, Neural Networks, Backpropagation.
Chapter-11, Chapter-12 Bishop Chapter 14,
stacking paper
Rojas Chapter 7.
Efficient Backprop by LeCun et al.
19
Nov. 12

Backpropagation.
Chapter-12 Rojas Chapter 7.
20
Nov. 16

Training Deep Nets, Optimization.
Chapter-12, Chapter-13 [1], [2], [3]
Optimization-1, Optimization-2
No new reading.
21
Nov. 19

Optimization, ConvNets.
Chapter-13, Chapter-14 Optimization-1, Optimization-2
ConvNets
22
Nov. 23

ConvNets, MLE, MAP, Bayesian Learning.
Chapter-14, Chapter-15 ConvNets
Parameter Estimation
No new reading.
23
Nov. 26

Bayesian Linear Regression.
Chapter-15 Bishop section 3.3
24
Nov. 30

Kernel Methods and Gaussian Processes.
25
Dec. 03

Tips and Tricks.
26
Dec. 07

Frontiers in ML, What Next?

Reference Materials

  1. [HTF] Trevor Hastie, Robert Tibshirani and Jerome Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Available free online.
  2. [Bishop] Christopher Bishop. Pattern Recognition and Machine Learning.
  3. [Mitchell] Tom Mitchell. Machine Learning.
  4. [TSKK] Pang-Ning Tan, Michael Steinbach, Anuj Karpatne, Vipin Kumar. Introduction to Data Mining.
  5. [Rojas] Raul Rojas. Neural Networks.
  6. [GBC] Ian Goodfellow, Yoshua Bengio and Aaron Courville. Deep Learning. Available free online.