Over the past decade, deep neural networks have brought about major advances in computer vision, natural language modeling, protein structure prediction, and several other applications. These neural networks succeed despite having far more model parameters than training data. Classical machine learning theory suggests that overparametrized models will overfit, and cannot explain deep networks' low generalization error. In this talk I will survey recent theoretical developments that seek to better explain the performance of deep learning models, and present new results on the generalization ability of quantized neural networks and interpolating predictors.