Summer Course on Probability and Machine Learning

TTI-Japan | 2018

Rachit Nimavat

Pedro Savarese

This is a short (1 week) course aimed towards a formal introduction to the basics of probability theory and machine learning. The goal of the first half of the course is to make us comfortable and rigorously reason about probability arising in the context of computer science and machine learning. In the second half of the course, we will start with basics of Machine Learning and build our way up to Neural Networks.


Course Plan

Day 1: PyTorch Setup and Introduction to Probability

Day 2: Concentration Inequalities and MLE

Day 3: Generalization Error and Introduction to ML

Day 4: Regression and Neural Networks

Day 5: Advanced Neural Networks



Chapter 14 of Great Ideas in Theoretical Computer Science by Anil Ada and Klaus Sutner

Lectures 10-17 of Mathematical Toolkit by Madhur Tulsiani

Lecture 6 of Introduction to the Theory of Machine Learning by Avrim Blum

TTIC Machine Learning Summer School by Suriya Gunasekar, Karl Stratos and Mesrob Ohannessian

Deep Learning Textbook by Ian Goodfellow, Yoshua Bengio and Aaron Courville

Day 1

Day 2

MLE Codes

Please find the code we used for MLE for coins here. As a guideline for the homework, we are also uploading the code for estimating bias of a 6-sided dice here.

Toy Example | Stock Market

The goal of this homework is to predict the movement of a dummy stock-market. You will be given data points: . The data point corresponds to the movement of the stock-market on day. Assume that there are working hours, and corresponds to whether the market went up or down during hour on day. That is, if market went up, and if it went down.

Assume that market movements on different days are i.i.d. You may not assume anything about dependence of the market movement during a single day. For example, it may happen that if market went up in first hour, it will surely go down in the last hour irrespective of what happened in the middle three hours.

You can find the data for 10,000 days of market movement here. Your goal is to predict market movement on the last hour for the 10 days of data present here.

We will discuss the solution tomorrow.

Day 3

Toy Example | Stock Market

Naive Bayes Python Code
Naive Bayes Python Notebook
Naive Bayes Python Code View Online

Day 4

logreg_numpy python file

logreg_pytorch python file

feedforward python file

toy_problem python file

Day 5

conv python file

feedforward_imdb python file