日本語

Hiroki Naganuma / 長沼 大樹

Biography

I am a Research Engineer on NVIDIA’s CHEF team in Redmond, Washington, with a Visiting Assistant Professor appointment at Keio University beginning in late 2026.

My work focuses on making large-scale training of foundation models more efficient through optimization algorithms and distributed systems. At NVIDIA, I work on efficient pretraining algorithms and the pretraining of Nemotron models. I study how optimizer geometry, batch sizes, and learning rate schedules shape training dynamics, and how these insights can reduce computation and communication costs.

I received my Ph.D. in Computer Science from Université de Montréal and Mila in 2026, advised by Ioannis Mitliagkas. Previously, I conducted research at Meta Superintelligence Labs, mentored by Hao-Jun Michael Shi and Parameswaran Raman; at Google DeepMind, mentored by Atish Agarwala and George E. Dahl; and at Microsoft Research, mentored by Philipp Witte and Russell J. Hewett. I hold B.Eng. and M.Eng. degrees from Tokyo Institute of Technology, where I was advised by Rio Yokota and graduated as valedictorian of the School of Computing’s master’s program. I received support as a member of the inaugural cohort of Masason Foundation scholars and as an RBC Borealis Fellow.

CV (October 2026) / Detailed Biography

Current position

Research Engineer, NVIDIA, Convergence for High-Efficiency Formulations (CHEF), 2026-08–present. Redmond, WA, United States.

Appointment from late 2026

Visiting Assistant Professor, Keio University.

Education

Selected honor

Research

Large-scale optimization, distributed deep learning, and efficient training of large language models

Invited Talks

  • — Japan Seattle AI Innovation Meetup
  • — Institute of Science Tokyo
  • — Cyberdyne Inc.
  • — ZOZO Research
  • — LinkedIn
  • — IST Colloquium, Kyoto University — The Geometry of Efficient Training: Co-Designing Optimizers and Systems at Scale Slides · Announcement
  • — McGill AI Lab
  • — FY2025 Research Proposal-Based Use Results Meeting, The University of Osaka
  • — SB Intuitions
  • — NVIDIA
  • — CUNY-JSPS Forum on Artificial Intelligence
  • — The 10th Mathematical Optimization Seminar, Meiji University
  • — Spring Seminar Series, Ottawa Japanese Researchers Association
  • — The 25th Lecture Series, Japan Healthcare Innovation Hub
  • — Seminar Series, Montreal Academy Club
  • — The 4th Lecture Series, Seattle Japanese Research Community
  • — The 8th Mathematical Optimization Seminar, Meiji University
  • — RICOS × ZOZO NeurIPS 2022 Paper Reading Group — Trends on Distribution Shift in NeurIPS 2022 Slides
  • — Montréal Machine Learning and Optimization
  • — The 54th AI Seminar, ABCI Grand Challenge — An Empirical Study on the Effect of Optimizers on OOD Generalization Slides · Web site
  • — The 6th Mathematical Optimization Seminar, Meiji University
  • — Summer Seminar Series, The Japanese Graduate Student Association in the United States (JGSAU) — Towards CS PhD in Canada Slides
  • — The 78th Artificial Intelligence Seminar: AI Trend – Top Conference Report Session (NeurIPS 2019) — Trends in Deep Learning Theory at NeurIPS 2019 Slides

Teaching & Mentoring

Teaching and service

  • — Visiting Assistant Professor, Keio University
  • — Admission Committee Member, Mila supervision (Ph.D. and M.Sc. admissions for Fall 2026)
  • — Teaching Volunteer, Kids Code Club
  • — Teaching Assistant, GTC Japan, Deep Learning Institute Workshop
  • — Teaching Assistant, Life is Tech! (coding camps for middle and high school students)

Students mentored

  • — Nishant Sharma, Indian Institute of Technology Delhi → Master's student at Mila (NeurIPS 2026 Workshop; under review)
  • — Tatsuhiro Nakamori, Ph.D. student at Keio University (ICML 2026 Workshop; under review)
  • — Mahdi Ghaznavi, Ph.D. student at Université de Montréal (ICLR 2026 Workshop (Oral, Travel Grant))
  • — Youssef Briki, Undergraduate student at Université de Montréal (ICML 2026)
  • — Youssef Fadhloun, Undergraduate student at National Institute of Applied Science and Technology (TMLR 2026)
  • — Kacem Mathlouthi, Undergraduate student at National Institute of Applied Science and Technology (TMLR 2026)
  • — Ansh Tiwari, Undergraduate student at California Institute of Technology (ICML 2026 Workshop; under review)
  • — Ganesh Talluri, High school student at BASIS Peoria → Arizona State University (TMLR 2026; ICML 2026 Workshop)
  • — Sora Nakai, Undergraduate student at Kyoto University → Visiting student at University of Washington (TMLR 2026)
  • — Laura Gomezjurado Gonzalez, Undergraduate student at Stanford University → Research intern at Microsoft Research (Redmond) (NeurIPS 2025 Workshop; ICLR 2026)
  • — Kotaro Yoshida, Undergraduate student at Tokyo Institute of Technology → Research intern at A*STAR Singapore and Sakana AI (TMLR 2024, 2025, 2026; ICLR 2025, 2026; Student Encouragement Award, IPSJ 2024)
  • — Takafumi Horie, Master's student at Ritsumeikan University → Ph.D. student at Kyoto University (NeurIPS 2024 Workshop; ICLR 2025, 2026)
  • — Haruka Kumagai, Master's student at the University of Tokyo → Visiting researcher at University of Trento (ACM MM 2023 Workshop; Travel Grant, NEC C&C Foundation)
  • — Keigo Tada, Master's student at Ritsumeikan University → Research scientist at Sony (IJCNN 2023; Student Encouragement Award, IPSJ 2023)

Awards and grants received by co-authors and mentees

  1. Marubun Research Promotion Foundation, International Exchange Grant (Apr. 2025)
  2. Travel Grant, Young Researchers' Meeting on Machine Learning (YAML 2024) (Sep. 2024)
  3. Travel Grant, The 19th Symposium of Young Researcher Association for NLP Studies (YANS 2024) (Sep. 2024)
  4. Student Encouragement Award, The 86th National Convention of IPSJ (IPSJ 2024) (Mar. 2024)
  5. NEC C&C Foundation, Grant for Researchers Attending International Conferences (Oct. 2023)
  6. FIT Encouragement Award, The 22nd Forum on Information Technology (FIT 2023) (Sep. 2023)
  7. International Information Science Foundation, Overseas Researcher Dispatch Grant (Jul. 2023)
  8. Hakuhodo DY Holdings Award, The 29th Annual Meeting of the Association for Natural Language Processing (NLP 2023) (Mar. 2023)
  9. Student Encouragement Award, The 85th National Convention of IPSJ (IPSJ 2023) (Mar. 2023)

Blog

Latest CV (PDF)