Lec. |
Date |
Topic |
Lecture Material |
Reference |
Readings |

Introduction to machine learning. |
announcements, slides, notes, scribbles. | Bishop section 1.1 | Computing Machinery and Intelligence Probability Review-1 Probability Review-2 Linear Algebra Review |
||

Introduction to machine learning. |
announcements, notes, scribbles. | Bishop section 1.1, Hastie section 2.3 |
|||

Statistical decision theory. |
notes, scribbles. | Hastie section 2.4, 2.5 | Singular Value Decomposition.Tutorial-1: Intro to Python. 19 Jan 5pm-6pm at Stewart Biology S3/3. |
||

Linear regression. |
notes, scribbles | Bishop section 3.1 | |||

Bias-variance tradeoff, Regularization |
notes, scribbles | Bishop section 1.5.5, 3.1.4, 3.2 Hastie section 3.4.1, 3.4.2 |
Linear Algebra review (7.1) in this pdf Lagrange Multiplier (Appendix E in Bishop) |
||

Linear Classification. |
notes | Bishop section 1.5.4, 4.1.1, 4.1.2 | |||

Indicator regression, PCA, LDA. |
notes, scribbles | Bishop section 4.1.3, 4.1.4 this pdf for pca. |
No new reading. Finish previous reading. | ||

Generalized Linear Models, GDA, Evaluation metrics for classification. |
notes, scribbles | Bishop section 4.2 | Tutorial-2: Intro to scikit-learn. 02 Feb 5pm-6pm at Stewart Biology S3/3. |
||

Naive Bayes, Logistic Regression. |
notes, scribbles | Bishop section 4.2.3, 4.3, 3.1.1. | Intro to convex optimization (page 91 to 102 in this book) | ||

Newton-Raphson method, Perceptron. |
notes, scribbles | Bishop section 4.3.3, 1.5.4, 4.1.7 Hastie section 4.5 |
|||

separating hyperplane, SVM. |
notes, scribbles | Bishop section 7.1 | Generative and discriminative classifiers by Tom Mitchell. | ||

SVM, Non-parametric Methods, Decision Trees. |
notes, scribbles | Bishop section 7.1.2, Tan et al section 3.3, Mitchell chapter 3 |
|||

Ensemble Learning: Bagging, Boosting. |
notes, scribbles | Bishop Chapter 14 | A Few Useful Things to Know about Machine Learning by Pedro Domingos | ||

Stacking, Neural Networks. |
notes, scribbles | Bishop Chapter 14, stacking paper |
|||

Neural Networks and Backpropagation |
notes, scribbles | Rojas Chapter 7. | Efficient Backprop by LeCun et al. | ||

Training deep neural nets |
notes, scribbles | [1], [2], [3] | Tutorial-3: Intro to PyTorch. 02 Mar 5pm-6pm at Stewart Biology S3/3. |
||

No class. |
|||||

No class. |
|||||

ConvNets, RNNs. |
notes, scribbles | ConvNets, Goodfellow Chapter 10. |
Optimization-1, Optimization-2 LSTMs, ConvNets |
||

RNNs, Optimization. |
notes, scribbles | Goodfellow Chapter 10, Optimization-1, Optimization-2 |
|||

MLE, MAP, Bayesian Learning. |
notes | Parameter Estimation | Bishop section 2.3 | ||

Bayesian Linear Regression, Gaussian Processes. |
|||||

Clustering, K-Means, DBScan. |
|||||

GMM, EM Algorithm. |
|||||

Mid-term review. |
Mid-term on April 04 from 5:30pm to 8:30pm. | ||||

Factor Analysis, ICA, CCA. |
|||||

Course Summary and Practical Tips. |
|||||

Frontiers in ML, What Next?. |

- [Hastie] Trevor Hastie, Robert Tibshirani and Jerome Friedman.
*The Elements of Statistical Learning: Data Mining, Inference, and Prediction.* - [Bishop] Christopher Bishop.
*Pattern Recognition and Machine Learning.* - [Mitchell] Tom Mitchell.
*Machine Learning.* - [Tan et al.] Pang-Ning Tan, Michael Steinbach, Anuj Karpatne, Vipin Kumar.
*Introduction to Data Mining.* - [Rojas] Raul Rojas.
*Neural Networks.* - [Goodfellow] Ian Goodfellow, Yoshua Bengio and Aaron Courville.
*Deep Learning.*