Mathematical Toolkit - Autumn 2019


TTIC 31150/CMSC 31150

T Th 9:30 - 10:50, TTIC 526

Discussion: W 4-5 pm, TTIC 526

Office Hours: T 11-12, TTIC 534 (Madhur)

                      F 2-3 pm, TTIC 4th floor (Shashank)

Instructor: Madhur Tulsiani

TA: Shashank Srivastava




The course is aimed at first-year graduate students and advanced undergraduates. The goal of the course is to collect and present important mathematical tools used in different areas of computer science. The course will mostly focus on linear algebra and probability. We intend to cover the following topics and examples:

  • Abstract linear algebra: vector spaces, linear transformations, Hilbert spaces, inner product, Gram-Schmidt orthogonalization, eigenvalues and eigenvectors, SVD.
  • Least squares, iterative solvers for systems of linear equations, spectral graph theory, perturbation and stability of eigenvalues
  • Discrete probability: random variables, Markov, Chebyshev and Chernoff-Hoeffding bounds
  • Gaussian variables, concentration inequalities, dimension reduction.
  • Spectral partitioning and clustering.
  • Additional topics (to be chosen from based on time and interest): Martingales, Markov Chains, Random Matrices, Tensors, Chaining Methods

The course will have 4 homeworks (40 percent), 2 quizzes (10 percent), a midterm (20 percent) and a final (30 percent).


There is no textbook for this course. Please see the "Resources" and "Recommended Books" sections below for some useful references.



Homeworks and Announcements


  • The first quiz will be in (the beginning of) class on October 17.
  • Homework 1 (Due on 10/24/19).
  • Homework 2 (Due on 11/14/19).
  • The second quiz will be in (the beginning of) class on November 21.
  • Homework 3 (Due on 11/30/19).
  • Homework 4 (Due on 12/8/19).


Problem Sheets from discussion sections




Lecture Plan and Notes


  • 10/1: Fields and vector spaces.
    [Notes]
  • 10/3: Bases of vector spaces, Lagrange interpolation.
    [Notes]
  • 10/8: Bases of infinite-dimensional spaces, Linear Transformations.
    [Notes]
  • 10/11: Eigenvalues and eigenvectors, Inner Product Spaces
    [Notes]
  • 10/15: More on Inner Product Spaces, Orthogonality and orthonormality
    [Notes]
  • 10/17: Adjoints of linear transformations, Self-adjoint operators
    [Notes]
  • 10/22: Real spectral theorem, Rayleigh quotients, positive semidefinite operators
    [Notes]
  • 10/24: Singular Value Decomposition
    [Notes]
  • 10/29: Singular Value Decomposition for matrices, Low-rank approximation
    [Notes]
  • 10/31: Least squares approximation using SVD, Gershgorin disc theorem
    [Notes]
  • 11/5: Solving systems of linear equations, stepest descent, conjugate gradient method
  • 11/7: Basics of probability: the finite case
    [Notes]
  • 11/12: Schwartz-Zippel lemma, polynomial identity testing, random variables and expectations
    [Notes]
  • 11/14: Goemetric random variables, Coupon collection, Randomized algorithm for Max 3-SAT
    [Notes]
  • 11/19: Independent sets in graphs, Markov’s and Chebyshev’s inequalities, threshold phenomena in random graphs
    [Notes]
  • 11/21: Chernoff-Hoeffding bounds
    [Notes]
  • 11/26: Probability over infinite spaces, sigma-fields and measurable functions, Gaussian random variables
    [Notes]


Resources




Recommended Books


There is no textbook for this course. The following books may be useful as references: