Hiroki Naganuma

Overview

• 今読まなきゃと思ってる Paper List を列挙する
• 今やってるプロジェクトとかも含めて研究分野ごとに分ける
• ここにまとめてるものをそのうち公開予定

Papers

Out-of-Distribution Generalization

• On Calibration and Out-of-domain Generalization
• Fishr: Invariant Gradient Variances for Out-of-distribution Generalization
• A Fine-Grained Analysis on Distribution Shift
• The Evolution of Out-of-Distribution Robustness Throughout Fine-Tuning
• Towards a Theoretical Framework of Out-of-Distribution Generalization

Calibration

• Can You Trust Your Model’s Uncertainty?
• Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning

Information Matrix

• On the Variance of the Fisher Information for Deep Learning
• ViViT: Curvature access through the generalized Gauss-Newton’s low-rank structure
• A Loss Curvature Perspective on Training Instability in Deep Learning

Training Dynamics

• SGD: The Role of Implicit Regularization, Batch-size and Multiple-epochs
• On the Explicit Role of Initialization on the Convergence and Generalization Properties of Overparametrized Linear Networks
• Beyond BatchNorm: Towards a General Understanding of Normalization in Deep Learning

Generative Adversarial Networks

• A Closer Look at the Optimization Landscapes of Generative Adversarial Networks

Hessian Spectrum Analysis

• Hessian based analysis of SGD for Deep Nets: Dynamics and Generalization
• Gradient Descent Happens in a Tiny Subspace
• An Investigation into Neural Net Optimization via Hessian Eigenvalue Density