INF8953CE: Machine Learning (Fall 2020)

Tentative Schedule for Fall 2020

Lecture videos are available here.

Lecture scribbles are available here.

Lec.
Date
Topic
Lecture Material
Reference
Readings
1
Aug. 31
Introduction to machine learning.		 Slides, chapter-01 Bishop section 1.1 Computing Machinery and Intelligence
Probability Review-1
Probability Review-2
Linear Algebra Review
2
Sep. 03
Regression and Overfitting.		 chapter-01 Bishop section 1.1,
HTF section 2.3
Sep. 07
HOLIDAY.		 Singular Value Decomposition.
3
Sep. 10
Overfitting, ML Pipeline, Classification.		 chapter-01 Bishop section 1.1,
HTF section 2.3
4
Sep. 14
Statistical Decision Theory, More on Linear Regression.		 Chapter-02, Chapter-03 HTF section 2.4, 2.5
Bishop section 3.1		 Linear Algebra review (7.1) in this pdf
Lagrange Multiplier (Appendix E in Bishop)
5
Sep. 17
Gradient Descent, Regularization.		 Chapter-03 Bishop section 3.1, 3.1.4
HTF section 3.4.1, 3.4.2
6
Sep. 21
Bias-variance Tradeoff, Linear Classification.		 Chapter-04, Chapter-05 Bishop section 1.5.5, 3.2, 1.5.4, 4.1.1, 4.1.2 Intro to convex optimization (page 91 to 102 in this book)
7
Sep. 24
Linear Classification.		 Chapter-05 Bishop section 1.5.4, 4.1.1, 4.1.2, 4.1.3
8
Sep. 28
Dimensionality reduction, PCA, LDA.		 Chapter-06 Bishop section 4.1.4
this pdf for pca.		 No new reading. Finish previous reading.
9
Oct. 01
GLM, GDA.		 Chapter-07 Bishop section 4.2
10
Oct. 05
GDA, Naive Bayes.		 Chapter-07 Bishop section 4.2 Bishop section 2.3
11
Oct. 08
Evaluation metric for classification, Logistic Regression.		 Chapter-07, Chapter-08 Bishop section 4.3
Oct. 12
Study Break.
Oct. 15
Study Break.
12
Oct. 19
Logistic Regression, Newton-Raphson Method, Perceptron.		 Chapter-08, Chapter-09 Bishop section 4.3, 3.1.1, 1.5.4, 4.1.7.
HTF section 4.5. 		Generative and discriminative classifiers by Tom Mitchell.
13
Oct. 22
Perceptron, Max-margin Classifier.		 Chapter-09 Bishop section 4.1.7, 7.1
HTF section 4.5
14
Oct. 26
Max-margin Classifier, SVM.		 Chapter-09 Bishop section 7.1 A Few Useful Things to Know about Machine Learning by Pedro Domingos
15
Oct. 29
Non-parametric methods, Decision Trees.		 Chapter-09, Chapter-10 Bishop section 7.1
TSKK section 3.3
Mitchell Chapter 3
16
Nov. 02
Decision Trees, Bagging, Random Forests, ERTs.		 Chapter-10, Chapter-11 TSKK section 3.3
Mitchell Chapter 3
Bishop Chapter 14 		No new reading. Finish previous reading.
17
Nov. 05
Boosting.		 Chapter-11 Bishop Chapter 14
18
Nov. 09
Stacking, Neural Networks, Backpropagation.		 Chapter-11, Chapter-12 Bishop Chapter 14,
stacking paper
Rojas Chapter 7. 		Efficient Backprop by LeCun et al.
19
Nov. 12
Backpropagation.		 Chapter-12 Rojas Chapter 7.
20
Nov. 16
Training Deep Nets, Optimization.		 Chapter-12, Chapter-13 [1], [2], [3]
Optimization-1, Optimization-2		 No new reading.
21
Nov. 19
Optimization, ConvNets.		 Chapter-13, Chapter-14 Optimization-1, Optimization-2
ConvNets
22
Nov. 23
ConvNets, MLE, MAP, Bayesian Learning.		 Chapter-14, Chapter-15 ConvNets
Parameter Estimation 		No new reading.
23
Nov. 26
Bayesian Linear Regression.		 Chapter-15 Bishop section 3.3
Nov. 30
CANCELLED.
24
Dec. 03
Tips and Tricks.		 Check scribbles.
25
Dec. 07
Frontiers in ML, What Next?		 Check scribbles.

Reference Materials

  1. [HTF] Trevor Hastie, Robert Tibshirani and Jerome Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Available free online.
  2. [Bishop] Christopher Bishop. Pattern Recognition and Machine Learning.
  3. [Mitchell] Tom Mitchell. Machine Learning.
  4. [TSKK] Pang-Ning Tan, Michael Steinbach, Anuj Karpatne, Vipin Kumar. Introduction to Data Mining.
  5. [Rojas] Raul Rojas. Neural Networks.
  6. [GBC] Ian Goodfellow, Yoshua Bengio and Aaron Courville. Deep Learning. Available free online.