COMP-551: Topics in Computer Science: Applied Machine Learning

Tentative Schedule for Fall 2018

Lec.
Date
Topic
Lecture Material
Reference
Readings
1
Sep. 05

Introduction to machine learning.
Logistics, Slides, Notes, Scribbles Bishop section 1.1 Computing Machinery and Intelligence
Probability Review-1
Probability Review-2
Linear Algebra Review
2
Sep. 10

Introduction to machine learning.
Notes, Scribbles Bishop section 1.1,
HTF section 2.3
3
Sep. 12

Statistical Decision Theory.
Notes, Scribbles HTF section 2.4, 2.5 Singular Value Decomposition.

Tutorial-1: Intro to Python. 14 Sep 6pm-8pm at ENGMC 304.
4
Sep. 17

Linear Regression, Gradient Descent.
Notes, Scribbles Bishop section 3.1
5
Sep. 19

Bias-variance tradeoff, Regularization.
Notes, Scribbles Bishop section 1.5.5, 3.1.4, 3.2
HTF section 3.4.1, 3.4.2
Linear Algebra review (7.1) in this pdf
Lagrange Multiplier (Appendix E in Bishop)
6
Sep. 24

Linear Classification.
Notes, Scribbles Bishop section 1.5.4, 4.1.1, 4.1.2
7
Sep. 26

Indicator Regression, PCA, LDA.
Notes, Scribbles Bishop section 4.1.3, 4.1.4
this pdf for pca.
Intro to convex optimization (page 91 to 102 in this book)
Oct. 01

Election Day.
Tutorial-2: Intro to scikit-learn. 03 Oct 6pm-8pm at ENGMC 204.
8
Oct. 03

GLMs, GDA, Evaluation metrics for classification.
Notes, Scribbles Bishop section 4.2 No new reading. Finish previous reading.
Oct. 08

Thanksgiving.
9
Oct. 10

Naive Bayes, Logistic Regression
Notes, Scribbles Bishop section 4.2.3, 4.3, 3.1.1. Generative and discriminative classifiers by Tom Mitchell.
10
Oct. 15

Logistic Regression, Newton-Raphson Method.
Notes, Scribbles Bishop section 4.3.3, 1.5.4.
11
Oct. 17

Perceptron.
Notes, no scribble. Bishop section 4.1.7
HTF section 4.5
Bishop section 2.3

Tutorial-2: Intro to scikit-learn. 22 Oct 6pm-8pm at ENGMC 204.
12
Oct. 22

Separating hyperplane, SVM.
Notes Bishop section 7.1
13
Oct. 24

SVM, Non parametric Methods.
Notes Bishop section 7.1.2 A Few Useful Things to Know about Machine Learning by Pedro Domingos
14
Oct. 29

Decision Trees.
Notes TSKK section 3.3
Mitchell Chapter 3
15
Oct. 31

Ensemble Learning: Bagging, Boosting.
Notes No new reading. Finish previous reading.
16
Nov. 05

Stacking, Neural Networks.
Notes
17
Nov. 07

Backpropagation Algorithm
Notes Efficient Backprop by LeCun et al.
Nov. 12

No Class.
18
Nov. 14

Training Deep Neural Nets, Optimization.
Notes Optimization-1, Optimization-2
LSTMs, ConvNets
19
Nov. 19

Convnets, RNNs, MLE, MAP, Bayesian Learning, Bayesian Linear Regression.
20
Nov. 21

Kernel Methods, Gaussian Processes.
No new reading. Focus on your project.
21
Nov. 26

Clustering, K-means, DBScan.
22
Nov. 28

Gaussian Mixture Models, EM Algorithm.
No new reading. Focus on your project.
23
Dec. 03

Course Summary and Practical Tips.
24
Dec. 04

Frontiers in ML, What Next?
No new reading. Focus on your project.

Reference Materials

  1. [HTF] Trevor Hastie, Robert Tibshirani and Jerome Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Available free online.
  2. [Bishop] Christopher Bishop. Pattern Recognition and Machine Learning.
  3. [Mitchell] Tom Mitchell. Machine Learning.
  4. [TSKK] Pang-Ning Tan, Michael Steinbach, Anuj Karpatne, Vipin Kumar. Introduction to Data Mining.
  5. [Rojas] Raul Rojas. Neural Networks.
  6. [GBC] Ian Goodfellow, Yoshua Bengio and Aaron Courville. Deep Learning. Available free online.