COMP-551: Topics in Computer Science: Applied Machine Learning

Tentative Schedule for Fall 2018

Lec.
Date
Topic
Lecture Material
Reference
Readings
1
Sep. 05
Introduction to machine learning.		 Logistics, Slides, Notes, Scribbles Bishop section 1.1 Computing Machinery and Intelligence
Probability Review-1
Probability Review-2
Linear Algebra Review
2
Sep. 10
Introduction to machine learning.		 Notes, Scribbles Bishop section 1.1,
HTF section 2.3
3
Sep. 12
Statistical Decision Theory.		 Notes, Scribbles HTF section 2.4, 2.5 Singular Value Decomposition.

Tutorial-1: Intro to Python. 14 Sep 6pm-8pm at ENGMC 304.
4
Sep. 17
Linear Regression, Gradient Descent.		 Notes, Scribbles Bishop section 3.1
5
Sep. 19
Bias-variance tradeoff, Regularization.		 Notes, Scribbles Bishop section 1.5.5, 3.1.4, 3.2
HTF section 3.4.1, 3.4.2		 Linear Algebra review (7.1) in this pdf
Lagrange Multiplier (Appendix E in Bishop)
6
Sep. 24
Linear Classification.		 Notes, Scribbles Bishop section 1.5.4, 4.1.1, 4.1.2
7
Sep. 26
Indicator Regression, PCA, LDA.		 Notes, Scribbles Bishop section 4.1.3, 4.1.4
this pdf for pca.		 Intro to convex optimization (page 91 to 102 in this book)
Oct. 01
Election Day.		 Tutorial-2: Intro to scikit-learn. 03 Oct 6pm-8pm at ENGMC 204.
8
Oct. 03
GLMs, GDA, Evaluation metrics for classification.		 Notes, Scribbles Bishop section 4.2 No new reading. Finish previous reading.
Oct. 08
Thanksgiving.
9
Oct. 10
Naive Bayes, Logistic Regression		 Notes, Scribbles Bishop section 4.2.3, 4.3, 3.1.1. Generative and discriminative classifiers by Tom Mitchell.
10
Oct. 15
Logistic Regression, Newton-Raphson Method.		 Notes, Scribbles Bishop section 4.3.3, 1.5.4.
11
Oct. 17
Perceptron.		 Notes, no scribble. Bishop section 4.1.7
HTF section 4.5		 Bishop section 2.3

Tutorial-2: Intro to scikit-learn. 22 Oct 6pm-8pm at ENGMC 204.
12
Oct. 22
Separating hyperplane, SVM.		 Notes, Scribbles Bishop section 7.1
13
Oct. 24
SVM, Non parametric Methods.		 Notes, Scribbles Bishop section 7.1.2 A Few Useful Things to Know about Machine Learning by Pedro Domingos
14
Oct. 29
Decision Trees.		 Notes, Scribbles TSKK section 3.3
Mitchell Chapter 3
15
Oct. 31
Ensemble Learning: Bagging, Boosting.		 Notes, Scribbles Bishop Chapter 14 No new reading. Finish previous reading.
16
Nov. 05
Stacking, Neural Networks.		 Notes, Scribbles Bishop Chapter 14,
stacking paper
17
Nov. 07
Backpropagation Algorithm		 Notes Rojas Chapter 7. Efficient Backprop by LeCun et al.
Nov. 12
No Class.
18
Nov. 14
Training Deep Neural Nets, Optimization.		 Notes [1], [2], [3]
Optimization-1, Optimization-2		 Optimization-1, Optimization-2
19
Nov. 19
Convnets		 Notes ConvNets
20
Nov. 21
Recurrent Neural Networks.		 Notes GBC Chapter 10. LSTMs, ConvNets
21
Nov. 26
MLE, MAP, Bayesian Learning.		 Notes Parameter Estimation
22
Nov. 28
Bayesian Linear Regression.		 Notes Bishop section 3.3 No new reading. Focus on your project.
23
Dec. 03
Kernel Methods, Gaussian Processes.		 Notes Bishop section 6.1, 6.2, 6.4
24
Dec. 04
Frontiers in ML, What Next?		 Notes No new reading. Focus on your project.

Reference Materials

  1. [HTF] Trevor Hastie, Robert Tibshirani and Jerome Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Available free online.
  2. [Bishop] Christopher Bishop. Pattern Recognition and Machine Learning.
  3. [Mitchell] Tom Mitchell. Machine Learning.
  4. [TSKK] Pang-Ning Tan, Michael Steinbach, Anuj Karpatne, Vipin Kumar. Introduction to Data Mining.
  5. [Rojas] Raul Rojas. Neural Networks.
  6. [GBC] Ian Goodfellow, Yoshua Bengio and Aaron Courville. Deep Learning. Available free online.