Hiroki Naganuma
Research Engineer, NVIDIAVisiting Assistant Prof, Keio University
Hiroki Naganuma is a Research Engineer at NVIDIA, where he works on efficient pretraining algorithms and the pretraining of Nemotron models, and has a Visiting Assistant Professor appointment at Keio University beginning in late 2026. His research covers large-scale optimization, distributed deep learning, and efficient training of large language models.
Ph.D. thesis: Toward Efficient and Scalable Optimization: Theoretical Insights and Practical Challenges
hiroki11x@gmail.com · ORCID · Google Scholar · researchmap · J-GLOBAL
News
- — Joined NVIDIA in Redmond, WA as a Research Engineer on the Convergence for High-Efficiency Formulations (CHEF) team, working on efficient pretraining algorithms and Nemotron pretraining.
- — Generalization Measures under Controlled Covariate Shift: A Regime-Aware Benchmark was accepted to TMLR.
- — Presented Adaptive Batch Sizes Using Non-Euclidean Gradient Noise Scales for Stochastic Sign and Spectral Descent at ICML 2026 in Seoul.
- — Invited talks in Japan at the Kyoto University IST Colloquium, LinkedIn, ZOZO Research, Cyberdyne, and Institute of Science Tokyo.
- — Defended my Ph.D. thesis at Université de Montréal / Mila. Thesis: Toward Efficient and Scalable Optimization: Theoretical Insights and Practical Challenges.
- — What do near-optimal learning rate schedules look like? (with Google DeepMind) was accepted to TMLR.
- — Received the Nippon Broadcasting Award at the 39th Japan Advanced Technology Award.
- — Received the UJA Young Investigator Award 2026.
- — Two papers accepted to ICLR 2026: On Fairness of Task Arithmetic and DiTaC.
Research areas
Selected and recent publications
- Understanding Stability of Outer Nesterov Momentum on Stochastic Pseudo-Gradients — arXiv (2026)
- Which Geometry on Which Layer? Measured Sign-Spectral Assignment in Mixed-Optimizer Training — arXiv (2026)
- Generalization Measures under Controlled Covariate Shift: A Regime-Aware Benchmark — TMLR 2026 (2026)
- Adaptive Batch Sizes Using Non-Euclidean Gradient Noise Scales for Stochastic Sign and Spectral Descent — ICML 2026 (2026)
- What do near-optimal learning rate schedules look like? — TMLR 2026 (2026)
- Orth-Dion: Eliminating Geometric Mismatch in Distributed Low-Rank Spectral Optimization — Under Review (2026)
- The Geometry of Spectral Gradient Descent: Layerwise Criteria for SignSGD vs SpecSGD — ICLR 2026 Workshop (2026)
- On Fairness of Task Arithmetic: The Role of Task Vectors — ICLR 2026 (2026)
- DiTaC: Conditioning Task Vectors via Distillation for Robust Model Merging — ICLR 2026 (2026)
- Pseudo-Asynchronous Local SGD: Robust and Efficient Data-Parallel Training — TMLR 2025 (2025)