COMP-551-001: Topics in Computer Science: Applied Machine Learning

Tentative Schedule for Winter 2018

Lec.
Date
Topic
Lecture Material
Reference
Readings
1
Jan. 9
Introduction to machine learning.		 announcements, slides, notes, scribbles. Bishop section 1.1 Computing Machinery and Intelligence
Probability Review-1
Probability Review-2
Linear Algebra Review
2
Jan. 11
Introduction to machine learning.		 announcements, notes, scribbles. Bishop section 1.1,
Hastie section 2.3
3
Jan. 16
Statistical decision theory.		 notes, scribbles. Hastie section 2.4, 2.5 Singular Value Decomposition.

Tutorial-1: Intro to Python. 19 Jan 5pm-6pm at Stewart Biology S3/3.
4
Jan. 18
Linear regression.		 notes, scribbles Bishop section 3.1
5
Jan. 23
Bias-variance tradeoff, Regularization		 notes, scribbles Bishop section 1.5.5, 3.1.4, 3.2
Hastie section 3.4.1, 3.4.2		 Linear Algebra review (7.1) in this pdf
Lagrange Multiplier (Appendix E in Bishop)
6
Jan. 25
Linear Classification.		 notes Bishop section 1.5.4, 4.1.1, 4.1.2
7
Jan. 30
Indicator regression, PCA, LDA.		 notes, scribbles Bishop section 4.1.3, 4.1.4
this pdf for pca.		 No new reading. Finish previous reading.
8
Feb. 1
Generalized Linear Models, GDA, Evaluation metrics for classification.		 notes, scribbles Bishop section 4.2 Tutorial-2: Intro to scikit-learn. 02 Feb 5pm-6pm at Stewart Biology S3/3.
9
Feb. 6
Naive Bayes, Logistic Regression.		 notes, scribbles Bishop section 4.2.3, 4.3, 3.1.1. Intro to convex optimization (page 91 to 102 in this book)
10
Feb. 8
Newton-Raphson method, Perceptron.		 notes, scribbles Bishop section 4.3.3, 1.5.4, 4.1.7
Hastie section 4.5
11
Feb. 13
separating hyperplane, SVM.		 notes, scribbles Bishop section 7.1 Generative and discriminative classifiers by Tom Mitchell.
12
Feb. 15
SVM, Non-parametric Methods, Decision Trees.		 notes, scribbles Bishop section 7.1.2,
Tan et al section 3.3,
Mitchell chapter 3
13
Feb. 20
Ensemble Learning: Bagging, Boosting.		 notes, scribbles Bishop Chapter 14 A Few Useful Things to Know about Machine Learning by Pedro Domingos
14
Feb. 22
Stacking, Neural Networks.		 notes, scribbles Bishop Chapter 14,
stacking paper
15
Feb. 27
Neural Networks and Backpropagation		 notes, scribbles Rojas Chapter 7. Efficient Backprop by LeCun et al.
16
Mar. 1
Training deep neural nets		 notes, scribbles [1], [2], [3] Tutorial-3: Intro to PyTorch. 02 Mar 5pm-6pm at Stewart Biology S3/3.
_
Mar. 6
No class.
_
Mar. 8
No class.
17
Mar. 13
ConvNets, RNNs.		 notes, scribbles ConvNets,
Goodfellow Chapter 10.		 Optimization-1, Optimization-2
LSTMs, ConvNets
18
Mar. 15
RNNs, Optimization.		 notes, scribbles Goodfellow Chapter 10,
Optimization-1, Optimization-2
19
Mar. 20
MLE, MAP, Bayesian Learning.		 notes, scribbles Parameter Estimation Bishop section 2.3
20
Mar. 22
Bayesian Linear Regression		 notes, scribbles Bishop section 3.3
21
Mar. 27
Kernel Methods, Gaussian Processes.		 notes, scribbles Bishop section 6.1, 6.2, 6.4 No new reading. Prepare for mid-term.
22
Mar. 29
Clustering, K-Means, DBScan.		 slides Tan et al. section 7.1, 7.2, 7.4, 7.5.
_
Apr. 3
Mid-term review.		 Mid-term on April 04 from 5:30pm to 8:30pm.
23
Apr. 5
Gaussian Mixture Models, EM Algorithm.		 notes, scribbles Bishop section 2.3.9, 9.2
24
Apr. 10
Course Summary and Practical Tips.		 notes, slides No new reading. Focus on the project.
25
Apr. 12
Frontiers in ML, What Next?.		 notes, slides, scribbles

References

  1. [Hastie] Trevor Hastie, Robert Tibshirani and Jerome Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction.
  2. [Bishop] Christopher Bishop. Pattern Recognition and Machine Learning.
  3. [Mitchell] Tom Mitchell. Machine Learning.
  4. [Tan et al.] Pang-Ning Tan, Michael Steinbach, Anuj Karpatne, Vipin Kumar. Introduction to Data Mining.
  5. [Rojas] Raul Rojas. Neural Networks.
  6. [Goodfellow] Ian Goodfellow, Yoshua Bengio and Aaron Courville. Deep Learning.