Biography
Hiroki Naganuma is a Research Engineer (Deep Learning Software Engineer) at NVIDIA and has a Visiting Assistant Professor appointment at Keio University beginning in October 2026. He works on large-scale optimization, distributed deep learning, and efficient training of large language models.
Current position
Research Engineer (Deep Learning Software Engineer), NVIDIA, Convergence for High-Efficiency Formulations (CHEF), 2026-08–present. Redmond, WA, United States.
Appointment from October 2026
Visiting Assistant Professor, Keio University.
Education
- Ph.D. — Doctor of Philosophy in Computer Science, Université de Montréal, 2026
Ph.D. thesis: Toward Efficient and Scalable Optimization: Theoretical Insights and Practical Challenges - M.Eng. — Master of Engineering in Computer Science, Tokyo Institute of Technology, 2019
- B.Eng. — Bachelor of Engineering in Computer Science, Tokyo Institute of Technology, 2017
Selected honor
- Nippon Broadcasting Award, the 39th Japan Advanced Technology Award (2026)
- UJA Young Investigator Award 2026 (United Japanese Researchers Abroad) (2026)
- Silver Reviewer Award, ICML 2026 (2026)
- Best Reviewer Award, AISTATS 2025 (2025)
- Top Reviewer Award, NeurIPS 2024 (2024)
- JHPCN Emerging Research Award, JHPCN Symposium 2024 (2024)
- Minister of Internal Affairs and Communications Award, Japan Campus Grand Prix (2022)
- National Winner and Japan Representative, James Dyson Award 2018 (2018)
Research
Large-scale optimization, distributed deep learning, and efficient training of large language models
Blog
- Qiita : Tech Blog in Japanese
- Understainding Implementation of Caffe (Convolutional Architecture for Fast Feature Embedding) [Japanese Blog]
- Introduction and Features of Apache Spark [Japanese Blog]
- How to use NVIDIA Visual Profiler [Japanese Blog]
- DynamoDB on local environment [Japanese Blog]
- Art work using openFrameworks and Twitter Streaming APIs [Japanese Blog]
- Medium
- Understanding optimization in deep learning by analyzing trajectories of gradient descent [Japanese Article]
- Is Optimization a Sufficient Language for Understanding Deep Learning? [Japanese Article]
- Understanding implicit regularization in deep learning by analyzing trajectories of gradient descent [Japanese Article]
- How to Escape Saddle Points Efficiently [Japanese Article]
- note
- North American ML PhD Industry Job Search: A Personal Account [Japanese Article]
- Surviving a CS PhD in North America (2025): Lessons from Canada and U.S. Research Internships [Japanese Article]
- SlideShare
- Survey of Recent Deep Learning with Low Precision [English Slide]
- Speaker Deck
- The Geometry of Efficient Training From Optimization History to Trajectory and Optimizer Geometry [English Slide]
- Trends in Deep Learning Theory at NeurIPS 2019 [Japanese Slide]
- Towards CS PhD in Canada [Japanese Slide]
- An Empirical Study on the Effect of Optimizers on OOD Generalization / ABCI Grand Challenge [English Slide] [Web Site]
- Trends on Distribution Shift in NeurIPS2022
- Public Article / Presentation / Podcast
- Article - July 2025 / Nikkei Robotics: New Regularization Technique from Institute of Science Tokyo and ZOZO Makes It Easier to Add and Remove Tasks from Neural Networks [Japanese Article]
- Personal Blog / github.io
- Scaling Laws [Japanese Article]
- Natural Gradient Descent [Japanese Article]
- GAN: Generative Adversarial Networks [Japanese Article]
- Out-of-Distribution Generalization [Japanese Article]
- Calibration [English Article]
- Automatic Differentiation [English Article]