This is a short (1 week) course aimed towards a formal introduction to the basics of probability theory and machine learning. The goal of the first half of the course is to make us comfortable and rigorously reason about probability arising in the context of computer science and machine learning. In the second half of the course, we will start with basics of Machine Learning and build our way up to Neural Networks.

- Python, PyTorch and NumPy setup
- Definitions, Events, Conditional Probability, Chain Rule, Bayes Rule, Independent Events
- Random Variables, Expectation, Linearity of Expectation, Independent Random Variables, Markov Inequality

- Coin Toss
- Maximum Likelihood Estimator for Biased Coin Tosses and Dice Rolls
- How to Differentiate between two Biased Coins?

- Please Validate your Hypothesis
- How many Samples do You Need to Learn Bias?
- Machine Learning: Motivation and Introduction
- Naive Bayes
- slides

- Logistic Regression
- Optimization and Model Selection
- Feedforward Neural Networks
- Backpropagation
- slides

- Convolutional Neural Networks
- Coding Session
- slides

Chapter 14 of Great Ideas in Theoretical Computer Science by Anil Ada and Klaus Sutner

Lectures 10-17 of Mathematical Toolkit by Madhur Tulsiani

Lecture 6 of Introduction to the Theory of Machine Learning by Avrim Blum

TTIC Machine Learning Summer School by Suriya Gunasekar, Karl Stratos and Mesrob Ohannessian

Deep Learning Textbook by Ian Goodfellow, Yoshua Bengio and Aaron Courville

The goal of this homework is to predict the movement of a dummy stock-market. You will be given data points: . The data point corresponds to the movement of the stock-market on day. Assume that there are working hours, and corresponds to whether the market went up or down during hour on day. That is, if market went up, and if it went down.

Assume that market movements on different days are i.i.d. You may not assume anything about dependence of the market movement during a single day. For example, it may happen that if market went up in first hour, it will surely go down in the last hour irrespective of what happened in the middle three hours.

You can find the data for 10,000 days of market movement here. Your goal is to predict market movement on the last hour for the 10 days of data present here.

We will discuss the solution tomorrow.

Naive Bayes Python Code

Naive Bayes Python Notebook

Naive Bayes Python Code View Online

Data

logreg_numpy python file

logreg_pytorch

logreg_pytorch python file

feedforward

feedforward python file

toy_problem

toy_problem python file

conv python file

feedforward_imdb

feedforward_imdb python file

dataset