Mathematical Toolkit - Autumn 2016


TTIC 31150/CMSC 31150

T Th 9 - 10:20, TTIC Room 530

Discussion: W 4-5 pm, TTIC Room 530

Instructor: Madhur Tulsiani

TA: Blake Woodworth



 

The course is aimed at first-year graduate students and advanced undergraduates. The goal of the course is to collect and present important mathematical tools used in different areas of computer science. The course will mostly focus on linear algebra and probability. We intend to cover the following topics and examples:

  • Abstract linear algebra: vector spaces, linear transformations, Hilbert spaces, inner product, Gram-Schmidt orthogonalization, eigenvalues and eigenvectors, SVD.
  • Least squares, iterative solvers for systems of linear equations, spectral graph theory, perturbation and stability of eigenvalues
  • Discrete probability: random variables, Markov, Chebyshev and Chernoff-Hoeffding bounds
  • Gaussian variables, concentration inequalities, dimension reduction.
  • Spectral partitioning and clustering.
  • Additional topics (to be chosen from based on time and interest): Martingales, Markov Chains, Random Matrices, Tensors

The course will have 4 homeworks (40 percent), 2 quizzes (10 percent), a midterm (20 percent) and a final (30 percent).


There is no textbook for this course. Useful references for each topic will be listed on this page as the course proceeds.



Homeworks and Announcements

  • The first quiz will be on Thursday, October 13.
  • Homework 1 (Due on 10/20/16).
  • Homework 2 (Due on 11/03/16).
  • The Midterm will be held in the class on Tuesday, November 1.
  • The second quiz will be on Tuesday, November 15.
  • Homework 3 (Due on 11/22/16).
  • Homework 4 (Due on 12/02/16).


Problem Sheets from discussion sections

[Discussion 3]     [Discussion 4]     [Discussion 5]     [Discussion 6]


Lecture Plan and Notes

  • 9/27: Fields and vector spaces.
    [Notes]
  • 9/29: Bases of vector spaces, Lagrange interpolation.
    [Notes]
  • 10/4: Linear transformations, Eigenvalues and eigenvectors.
    [Notes]
  • 10/6: Inner product spaces, orthogonalization, Parseval's identity, adjoint of an operator
    [Notes]
  • 10/11: Existence and uniqueness of adjoint, spectral theorem for self-adjoint operators.
    [Notes]
  • 10/13: Existence of eigenvalues, Rayleigh quotients, min-max characterizations of eigenvalues.
    [Notes]
  • 10/18: Singular value decomposition, applications to low-rank approximation of matrices.
    [Notes]
  • 10/20: Approximating point sets by low-dimensional subspaces, Gershgorin disc theorem, application to proving bounds for the restricted isometry property, Solving systems of linear equations, Gaussian elimination.
    [Notes]
  • 10/25: Iterative methods for solving sparse linear systems, steepest descent.
    [Notes]
  • 10/27: Krylov subspaces and the conjugate gradient method for solving sparse systems, matrices associated with graphs.
    [Notes]
  • 11/1: Midterm
  • 11/3: Basics of probability and discrete random variables, Schwartz-Zippel lemma, polynomial identity testing.
    [Notes]
  • 11/8: Expectations of random variables, coupon collection, Markov's and Chebyshev's inequalities, thresholds in random graphs.
    [Notes]
  • 11/10: Threshold phenomena wrap-up, Chernoff-Hoeffding bounds, randomized load balancing.
    [Notes]
  • 11/15: The power of two random choices.
    [Notes]
  • 11/17: Probability over infinite spaces, sigma-fields and measurable functions, Gaussian random variables, Johnson-Lindenstrauss lemma.
    [Notes]
  • 11/22: Convexity and Jensen's inequality, Chernoff bound for bounded random variables, Chernoff bound for sums of random p.s.d. matrices.
  • 11/29: Chernoff bounds for matrices (wrap-up), application to connectivity threshold for random graphs, proof of Cheeger's inequality.


Resources



Recommended Books

There is no textbook for this course. The following books may be useful as references:
  • The Linear Algebra a Beginning Graduate Student Ought to Know
    (by Jonathan S. Golan)
  • Foundations of Data Science (by Hopcroft and Kannan)
  • Probability and Computing (by Mitzenmacher and Upfal)
  • The Probabilistic Method (by Alon and Spencer)