Machine Learning

INF8953CE - Machine Learning (Fall 2020)

Course Description

This course provides a rigorous introduction to the field of machine learning (ML). The aim of the course is not just to teach how to use ML algorithms but also to explain why, how, and when these algorithms work. The course introduces fundamental algorithms in supervised learning and unsupervised learning from the first principles. The course, while covering several problems in machine learning like regression, classification, representation learning, dimensionality reduction, will introduce the core theory, which unifies all the algorithms.

This course will be offered in English. However, the students in this course can submit in English or French any written work that is to be graded.

Quebec university students from outside Polytechnique Montreal can register for the course via Inter-University Transfer Authorization.

General Information

When:	Mondays 12:45-14:45 and Thursdays 16:45-17:45
Where:	Online (link available in Moodle)
Instructor:	Sarath Chandar Office hours: Mondays 14:45-15:45.
Teaching Assistants:	Abdelrahman Zayed Office hours: Wednesdays 13:00-14:00. Charan Reddy Office hours: Thursdays 15:00-16:00. Lawrence Abdulnour Office hours: Fridays 13:00-14:00. Mohammad Yaghoubi Office hours: None. Rémi Piché-Taillefer Office hours: Tuesdays 16:00-17:00.
Schedule:	Available Here.
Lecture Notes:	Available here.
Assignments and Tutorials:	Available here.

Tentative Course content:

Introduction - Prediction - Statistical Decision Theory - Linear Regression - Non-linear Regression - Bias-variance tradeoff - Linear Classification - Indicator Regression - PCA - LDA - QDA - GDA - Naive Bayes - Logistic Regression - Perceptron - Separating Hyperplanes - SVM - Decision Trees - ensemble learning - bagging - boosting - stacking - Neural Networks - Backpropagation - Training Deep Neural Nets - Optimization Methods - Convnets - RNNs - Estimation Theory - Maximum Likelihood Estimation - Maximum A Posteriori Estimation - Bayesian Learning - Bayesian Linear Regression - Kernel Methods - Gaussian Process - Clustering - K-means - GMM - EM Algorithm - Computational Learning Theory - Frontiers in ML.

Reference Materials

Lecture notes will be available from the course web page. The course is based on the following references.

[HTF] Trevor Hastie, Robert Tibshirani and Jerome Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Available free online.
[Bishop] Christopher Bishop. Pattern Recognition and Machine Learning.
[Mitchell] Tom Mitchell. Machine Learning.
[TSKK] Pang-Ning Tan, Michael Steinbach, Anuj Karpatne, Vipin Kumar. Introduction to Data Mining.
[Rojas] Raul Rojas. Neural Networks.
[GBC] Ian Goodfellow, Yoshua Bengio and Aaron Courville. Deep Learning. Available free online

Prerequisites

Basic knowledge of Probability Theory/statistics (MTH2303 or equivalent), calculus, and linear algebra (MTH1070 or equivalent) is required.

The course is intended for hard-working, technically skilled, highly motivated students. Participants will be expected to display initiative, creativity, scientific rigour, critical thinking, and good communication skills.

Useful Online Courses covering the Prerequisites

While I do not expect you to know everything from the following courses, I recommend you to do these video courses at some point in the future if you are serious about doing Machine Learning.

Prof. Gilbert Strang's video lectures on linear algebra.
Prof. John Tsitsiklis's video lectures on Applied Probability.
Prof. Krishna Jagannathan's video lectures on Probability Theory.
Prof. Deepak Khemani's video lectures on Artificial Intelligence.

Video Recordings

The lectures and tutorials will be recorded and released to the public. By registering for the course, you agree for the recording and release of videos.

If you do not want your video to be visible while asking questions, you can turn off your video. If you do not want your audio to be part of the recording, you can ask your questions in chat or during office hours.

Programming Language

We will use Python 3 in all the assignments. All the programming tutorials will be only in Python 3.

Evaluation Criteria

The class grade will be based on the following components:

3 Theory/Programming assignments (individual) - 45%
Kaggle competition (team of 3) - 20%
One endterm examination - 35%

We will use gradescope for all the assignments and projects. More detailed instructions on how to use gradescope will be released in the beginning of the course.

Late Submissions

If you submit your assignments and competition reports after the deadline, we will follow the following penalty scheme:

You will be penalized 5% if your submission is within 24 hours (1 day) from the deadline.
You will be penalized 10% if you submission is after 24 hours from the deadline and within 48 hours (2 days) from the deadline.
You will be penalized 20% if you submission is after 48 hours from the deadline and within 72 hours (3 days) from the deadline.
You cannot submit your assignments/reports after 72 hours from the deadline.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.