Publications

[*] indicates alphabetical ordering of author names.

Papers

  • Teacher's pet: understanding and mitigating biases in distillation
    Michal Lukasik, Srinadh Bhojanapalli, Aditya Krishna Menon, Sanjiv Kumar
    Preprint 2021
    [Arxiv]

  • Eigen Analysis of Self-Attention and its Reconstruction from Partial Computation [*]
    Srinadh Bhojanapalli, Ayan Chakrabarti, Himanshu Jain, Sanjiv Kumar, Michal Lukasik, Andreas Veit
    Preprint 2021
    [Arxiv]

  • Demystifying the Better Performance of Position Encoding Variants for Transformer
    Pu-Chin Chen*, Henry Tsai*, Srinadh Bhojanapalli*, Hyung Won Chung, Yin-Wen Chang, Chun-Sung Ferng
    EMNLP 2021
    [Arxiv]

  • Understanding Robustness of Transformers for Image Classification [*]
    Srinadh Bhojanapalli, Ayan Chakrabarti, Daniel Glasner, Daliang Li, Thomas Unterthiner, Andreas Veit
    ICCV 2021
    [Arxiv]

  • On the Reproducibility of Neural Network Predictions
    Srinadh Bhojanapalli, Kimberly Wilber, Andreas Veit, Ankit Singh Rawat, Seungyeon Kim, Aditya Menon, Sanjiv Kumar
    Preprint 2021
    [Arxiv]

  • Modifying Memories in Transformer Models
    Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix Yu, Sanjiv Kumar
    Preprint 2020
    [Arxiv]

  • Coping with Label Shift via Distributionally Robust Optimisation
    Jingzhao Zhang, Aditya Menon, Andreas Veit, Srinadh Bhojanapalli, Sanjiv Kumar, Suvrit Sra
    ICLR 2021
    [Openreview], [Arxiv]

  • An efficient nonconvex reformulation of stagewise convex optimization problems
    Rudy Bunel, Oliver Hinder, Srinadh Bhojanapalli, Krishnamurthy (Dj)Dvijotham
    Neurips 2020
    [Arxiv]

  • Semantic label smoothing for sequence to sequence problems
    Michal Lukasik, Himanshu Jain, Aditya Menon, Seungyeon Kim, Srinadh Bhojanapalli, Felix Yu and Sanjiv Kumar
    EMNLP 2020
    [Arxiv]

  • O(n) Connections are Expressive Enough: Universal Approximability of Sparse Transformers
    Chulhee Yun, Yin-Wen Chang, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar
    Neurips 2020
    [Arxiv].

  • Does label smoothing mitigate label noise?
    Michal Lukasik, Srinadh Bhojanapalli, Aditya Krishna Menon, Sanjiv Kumar
    ICML 2020
    [Arxiv].

  • Low-Rank Bottleneck in Multi-head Attention Models
    Srinadh Bhojanapalli, Chulhee Yun, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar
    ICML 2020
    [Arxiv].

  • Are Transformers universal approximators of sequence-to-sequence functions?
    Chulhee Yun, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank Reddi, Sanjiv Kumar
    ICLR 2020
    [Openreview].

  • Large Batch Optimization for Deep Learning: Training BERT in 76 minutes
    Yang You, Jing Li, Sashank Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, Kurt Keutzer, Cho-Jui Hsieh
    ICLR 2020
    [Openreview], [Arxiv].

  • The role of over-parametrization in generalization of neural networks
    Behnam Neyshabur, Zhiyuan Li, Srinadh Bhojanapalli, Yann LeCun, Nathan Srebro
    ICLR 2019
    [Openreview], [Arxiv].

  • Smoothed analysis for low-rank solutions to semidefinite programs in quadratic penalty form [*]
    Srinadh Bhojanapalli, Nicolas Boumal, Prateek Jain, Praneeth Netrapalli
    COLT 2018
    [Arxiv].

  • A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks
    Behnam Neyshabur, Srinadh Bhojanapalli, Nathan Srebro
    ICLR 2018
    [Arxiv].

  • Exploring Generalization in Deep Learning
    Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, Nathan Srebro
    NIPS 2017
    [Arxiv].

  • Implicit Regularization in Matrix Factorization
    Suriya Gunasekar, Blake Woodworth, Srinadh Bhojanapalli, Behnam Neyshabur, Nathan Srebro
    NIPS 2017

  • Stabilizing GAN Training with Multiple Random Projections
    Behnam Neyshabur, Srinadh Bhojanapalli, Ayan Chakrabarti
    preprint 2017, [Arxiv], [Project website]. [Arxiv], [slides].

  • Single Pass PCA of Matrix Products
    Shanshan Wu, Srinadh Bhojanapalli, Sujay Sanghavi, Alex Dimakis
    NIPS 2016
    [Arxiv], [SPARK code].

  • Global Optimality of Local Search for Low Rank Matrix Recovery
    Srinadh Bhojanapalli, Behnam Neyshabur, Nathan Srebro
    NIPS 2016
    [Arxiv].

  • Provable Burer-Monteiro factorization for a class of norm-constrained matrix problems
    Dohyung Park, Anastasios Kyrillidis, Srinadh Bhojanapalli, Constantine Caramanis, Sujay Sanghavi
    preprint 2016, [Arxiv].

  • Dropping Convexity for Faster Semi-definite Optimization
    Srinadh Bhojanapalli, Anastasios Kyrillidis, Sujay Sanghavi
    COLT 2016
    [Arxiv], [COLT], [slides].

  • A New Sampling Technique for Tensors
    Srinadh Bhojanapalli, Sujay Sanghavi
    preprint 2015, [Arxiv], [slides].

  • Tighter Low-rank Approximation via Sampling the Leveraged Element
    Srinadh Bhojanapalli, Prateek Jain, Sujay Sanghavi
    SODA 2015
    [Arxiv], [SODA], [slides].

  • Completing any Low-rank Matrix, Provably
    Yudong Chen, Srinadh Bhojanapalli, Sujay Sanghavi, Rachel Ward
    JMLR 2015
    [Arxiv], [JMLR].

  • Universal Matrix Completion
    Srinadh Bhojanapalli, Prateek Jain
    ICML 2014
    [Arxiv], [ICML], [slides], [video].

  • Coherent Matrix Completion
    Yudong Chen, Srinadh Bhojanapalli, Sujay Sanghavi, Rachel Ward
    ICML 2014
    [Arxiv], [ICML], [slides], [video].

PhD Thesis

Large Scale Matrix Factorization with Guarantees: Sampling and Bi-linearity [pdf]
UT Austin, 2015.