Lec. |
Date |
Topic |
Lecture Material |
Reference |
Readings |

Introduction to machine learning. |
Slides, chapter-01 | Bishop section 1.1 | Computing Machinery and Intelligence Probability Review-1 Probability Review-2 Linear Algebra Review |
||

Regression and Overfitting. |
chapter-01 | Bishop section 1.1, HTF section 2.3 |
|||

HOLIDAY. |
Singular Value Decomposition. | ||||

Overfitting, ML Pipeline, Classification. |
chapter-01 | Bishop section 1.1, HTF section 2.3 |
|||

Statistical Decision Theory, More on Linear Regression. |
Chapter-02, Chapter-03 | HTF section 2.4, 2.5 Bishop section 3.1 |
Linear Algebra review (7.1) in this pdf Lagrange Multiplier (Appendix E in Bishop) |
||

Gradient Descent, Regularization. |
Chapter-03 | Bishop section 3.1, 3.1.4 HTF section 3.4.1, 3.4.2 |
|||

Bias-variance Tradeoff, Linear Classification. |
Chapter-04, Chapter-05 | Bishop section 1.5.5, 3.2, 1.5.4, 4.1.1, 4.1.2 | Intro to convex optimization (page 91 to 102 in this book) | ||

Linear Classification. |
Chapter-05 | Bishop section 1.5.4, 4.1.1, 4.1.2, 4.1.3 | |||

Dimensionality reduction, PCA, LDA. |
Chapter-06 | Bishop section 4.1.4 this pdf for pca. |
No new reading. Finish previous reading. | ||

GLM, GDA. |
Chapter-07 | Bishop section 4.2 | |||

GDA, Naive Bayes. |
Chapter-07 | Bishop section 4.2 | Bishop section 2.3 | ||

Evaluation metric for classification, Logistic Regression. |
Chapter-07, Chapter-08 | Bishop section 4.3 | |||

Study Break. |
|||||

Study Break. |
|||||

Logistic Regression, Newton-Raphson Method, Perceptron. |
Chapter-08, Chapter-09 | Bishop section 4.3, 3.1.1, 1.5.4, 4.1.7. HTF section 4.5. |
Generative and discriminative classifiers by Tom Mitchell. | ||

Perceptron, Max-margin Classifier. |
Chapter-09 | Bishop section 4.1.7, 7.1 HTF section 4.5 |
|||

Max-margin Classifier, SVM. |
Chapter-09 | Bishop section 7.1 | A Few Useful Things to Know about Machine Learning by Pedro Domingos | ||

Non-parametric methods, Decision Trees. |
Chapter-09, Chapter-10 | Bishop section 7.1 TSKK section 3.3 Mitchell Chapter 3 |
|||

Decision Trees, Bagging, Random Forests, ERTs. |
Chapter-10, Chapter-11 | TSKK section 3.3 Mitchell Chapter 3 Bishop Chapter 14 |
No new reading. Finish previous reading. | ||

Boosting. |
Chapter-11 | Bishop Chapter 14 | |||

Stacking, Neural Networks, Backpropagation. |
Chapter-11, Chapter-12 | Bishop Chapter 14, stacking paper Rojas Chapter 7. |
Efficient Backprop by LeCun et al. | ||

Backpropagation. |
Chapter-12 | Rojas Chapter 7. | |||

Training Deep Nets, Optimization. |
Chapter-12, Chapter-13 | [1], [2], [3] Optimization-1, Optimization-2 |
No new reading. | ||

Optimization, ConvNets. |
Chapter-13, Chapter-14 | Optimization-1, Optimization-2 ConvNets |
|||

ConvNets, MLE, MAP, Bayesian Learning. |
Chapter-14, Chapter-15 | ConvNets Parameter Estimation |
No new reading. | ||

Bayesian Linear Regression. |
Chapter-15 | Bishop section 3.3 | |||

Kernel Methods and Gaussian Processes. |
|||||

Tips and Tricks. |
|||||

Frontiers in ML, What Next? |

- [HTF] Trevor Hastie, Robert Tibshirani and Jerome Friedman.
*The Elements of Statistical Learning: Data Mining, Inference, and Prediction.*Available free online. - [Bishop] Christopher Bishop.
*Pattern Recognition and Machine Learning.* - [Mitchell] Tom Mitchell.
*Machine Learning.* - [TSKK] Pang-Ning Tan, Michael Steinbach, Anuj Karpatne, Vipin Kumar.
*Introduction to Data Mining.* - [Rojas] Raul Rojas.
*Neural Networks.* - [GBC] Ian Goodfellow, Yoshua Bengio and Aaron Courville.
*Deep Learning.*Available free online.