Blake Woodworth

blakewoodworth (at) gmail (dot) com

About

Since October 2021, I have been a postdoctoral researcher with the SIERRA team at Inria, working with Francis Bach.

Before that, I was a PhD student in computer science at the Toyota Technological Institute at Chicago (TTIC) advised by Nati Srebro, and I graduated in Summer 2021. The primary focus of my PhD research was in the theory of optimization, with a particular emphasis on precisely understanding the oracle complexity of convex, non-convex, and distributed optimization problems. In addition to my work in optimization, I have also been interested in efforts to understand modern, highly overparametrized machine learning models through the lens of implicit regularization. Earlier in my PhD, I also enjoyed working on fairness in ML and on adaptive data analysis.

Before TTIC, I studied at Yale University where I received a B.S. in computer science, advised by Dan Spielman. At Yale, my coursework was spread evenly across the computer science, mathematics, and statistics departments; I was also a peer tutor for several programming-intensive computer science courses.

From September 2017-July 2019 I was supported by a NSF Graduate Research Fellowship, and from July 2019-August 2021, I was supported by a Google PhD Fellowship in machine learning.

My CV.

My Google Scholar page.

Publications

• Non-Convex Optimization with Certificates and Fast Rates Through Kernel Sums of Squares
  Blake Woodworth, Francis Bach, Alessandro Rudi
  arXiv, 2022

• The Minimax Complexity of Distributed Optimization
  Blake Woodworth
  PhD Thesis, 2021

• A Stochastic Newton Algorithm for Distributed Convex Optimization
  Brian Bullins, Kumar Kshitij Patel, Ohad Shamir, Nathan Srebro, Blake Woodworth
  NeurIPS, 2021

• An Even More Optimal Stochastic Optimization Algorithm: Minibatching and Interpolation Learning
  Blake Woodworth, Nathan Srebro
  NeurIPS, 2021

• On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent
  Shahar Azulay, Edward Moroshko, Mor Shpigel Nacson, Blake Woodworth, Nathan Srebro, Amir Globerson, Daniel Soudry
  ICML, 2021

• The Min-Max Complexity of Distributed Stochastic Convex Optimization with Intermittent Communication
  Blake Woodworth, Brian Bullins, Ohad Shamir, Nathan Srebro
  Best Paper Award
  COLT, 2021

• Mirrorless Mirror Descent: A More Natural Discretization of Riemannian Gradient Flow
  Suriya Gunasekar, Blake Woodworth, Nathan Srebro
  AISTATS, 2021

• Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy
  Edward Moroshko, Suriya Gunasekar, Blake Woodworth, Jason D. Lee, Nathan Srebro, Daniel Soudry
  NeurIPS, 2020

• Minibatch vs Local SGD for Heterogeneous Distributed Learning
  Blake Woodworth, Kumar Kshitij Patel, Nathan Srebro
  NeurIPS, 2020

• Is Local SGD Better than Minibatch SGD?
  Blake Woodworth, Kumar Kshitij Patel, Sebastian U. Stich, Zhen Dai, Brian Bullins, H. Brendan McMahan, Ohad Shamir, Nathan Srebro
  ICML, 2020
  Python code

• Kernel and Deep Regimes in Overparametrized Models
  Blake Woodworth, Suriya Gunasekar, Jason D. Lee, Edward Moroshko, Pedro Savarese, Itay Golan, Daniel Soudry, Nathan Srebro
  COLT, 2020

• The Gradient Complexity of Linear Regression
  Mark Braverman, Elad Hazan, Max Simchowitz, Blake Woodworth
  COLT, 2020

• Guaranteed Validity for Empirical Approaches to Adaptive Data Analysis
  Ryan Rogers, Aaron Roth, Adam Smith, Nathan Srebro, Om Thakkar, Blake Woodworth
  AISTATS, 2020

• Lower Bounds for Non-Convex Stochastic Optimization
  Yossi Arjevani, Yair Carmon, John C. Duchi, Dylan J. Foster, Nathan Srebro, Blake Woodworth
  Mathematical Programming, 2022

• Open Problem: The Oracle Complexity of Convex Optimization with Limited Memory
  Blake Woodworth, Nathan Srebro
  COLT, 2019

• The Complexity of Making the Gradient Small in Stochastic Convex Optimization
  Dylan Foster, Ayush Sekhari, Ohad Shamir, Nathan Srebro, Karthik Sridharan, Blake Woodworth
  Best Student Paper Award
  COLT, 2019

• Graph Oracle Models, Lower Bounds, and Gaps for Parallel Stochastic Optimization
  Blake Woodworth, Jialei Wang, Adam Smith, Brendan McMahan, and Nathan Srebro
  NeurIPS, 2018

• Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent Constraints
  Andrew Cotter, Maya Gupta, Heinrich Jiang, Nathan Srebro, Karthik Sridharan, Serena Wang, Blake Woodworth, and Seungil You
  FAT/ML 2018, ICML 2019

• The Everlasting Database: Statistical Validity at a Fair Price
  Blake Woodworth, Vitaly Feldman, Saharon Rosset, and Nathan Srebro
  NeurIPS, 2018

• Lower Bound for Randomized First Order Convex Optimization
  Blake Woodworth and Nathan Srebro
  arXiv, 2017

• Implicit Regularization in Matrix Factorization
  Suriya Gunasekar, Blake Woodworth, Srinadh Bhojanapalli, Behnam Neyshabur, and Nathan Srebro
  NeurIPS, 2017

• Learning Non-Discriminatory Predictors
  Blake Woodworth, Suriya Gunasekar, Mesrob I. Ohannessian, and Nathan Srebro
  COLT, 2017

• Tight Complexity Bounds for Optimizing Composite Objectives
  Blake Woodworth and Nathan Srebro
  NeurIPS, 2016

Work Experience

• Research Intern at Microsoft Research Cambridge, UK (Summer 2017)
  I worked with graph-structured neural networks (e.g. GG-NN) in collaboration with Ryota Tomioka and other members of the MSR Cambridge AMPNet team.

• Research Intern at Google Research Princeton, NJ (Summer 2019)
  I worked with Elad Hazan and Naman Agarwal on online learning and optimization algorithms.

Teaching Experience

• Teaching Assistant for TTIC 31120 - Statistical and Computational Learning Theory (Fall 2018)
  With Professor Nathan Srebro

• Teaching Assistant for TTIC 31070 - Convex Optimization (Winter 2018)
  With Professor Nathan Srebro

• Teaching Assistant for TTIC 31150 - Mathematical Toolkit (Fall 2016)
  With Professor Madhur Tulsiani

• Peer Tutor at Yale for CPSC 201, 223, and 323 (January 2013 - May 2015)
  With Professors Dana Angluin, James Aspnes, and Stanley Eisenstat

Graduate Coursework

• Introduction to Statistical Machine Learning (Fall 2015)
• Mathematical Toolkit (Fall 2015)
• Convex Optimization (Fall 2015)
• Algorithms (Winter 2016)
• Introduction to Computer Vision (Winter 2016)
• Probabilistic Graphical Models (Spring 2016)
• Park City Mathematics Institute (Summer 2016)
• Statistical and Computational Learning Theory (Fall 2016)
• Basic Functional Analysis (Winter 2017)
• Measure-Theoretic Probability I (Winter 2017)
• Unsupervised Learning and Data Analysis (Spring 2017)

Last Updated: April 22, 2022