# Research Overview — Hiroki Naganuma

> Research on efficient and scalable machine learning, connecting optimization theory with large-scale training systems.

Canonical HTML: https://hiroki11x.github.io/research/

## Scalable Pretraining and Distributed Learning

Efficient training of large language and foundation models through distributed, semi-synchronous, and large-batch methods.

Topics: large language model pretraining · distributed deep learning · large-batch training · high-performance computing

- [Pseudo-Asynchronous Local SGD: Robust and Efficient Data-Parallel Training](https://hiroki11x.github.io/publications/pseudo-asynchronous-local-sgd/) — TMLR 2025 (2025); Scalable Pretraining and Distributed Learning
- [Adaptive Batch Sizes Using Non-Euclidean Gradient Noise Scales for Stochastic Sign and Spectral Descent](https://hiroki11x.github.io/publications/adaptive-batch-sizes-using-non-euclidean-gradient-noise-scales/) — ICML 2026 (2026); Scalable Pretraining and Distributed Learning; Optimization for Machine Learning
- [Orth-Dion: Eliminating Geometric Mismatch in Distributed Low-Rank Spectral Optimization](https://hiroki11x.github.io/publications/orth-dion/) — Under Review (2026); Scalable Pretraining and Distributed Learning; Optimization for Machine Learning
- [Research notes](https://hiroki11x.github.io/posts/research_topics/)

## Optimization for Machine Learning

Optimization algorithms and training dynamics, including learning-rate schedules, non-Euclidean and spectral methods, Muon, and critical batch size analysis.

Topics: machine learning optimization · Muon optimizer · non-Euclidean optimization · learning-rate schedules · critical batch size

- [What do near-optimal learning rate schedules look like?](https://hiroki11x.github.io/publications/what-do-near-optimal-learning-rate-schedules-look-like/) — TMLR 2026 (2026); Optimization for Machine Learning
- [Convergence Bound and Critical Batch Size of Muon Optimizer](https://hiroki11x.github.io/publications/convergence-bound-and-critical-batch-size-of-muon/) — Under Review (2025); Scalable Pretraining and Distributed Learning; Optimization for Machine Learning
- [No Wrong Turns: The Simple Geometry Of Neural Networks Optimization Paths](https://hiroki11x.github.io/publications/no-wrong-turns-neural-network-optimization-paths/) — ICML 2024 (2024); Optimization for Machine Learning
- [Muon optimizer research note](https://hiroki11x.github.io/posts/research_topics/optimization/MUON/)

## Generalization and Foundation Models

Generalization, calibration, fairness, and model composition for pre-trained and foundation models under distribution shift.

Topics: foundation models · out-of-distribution generalization · model calibration · distribution shift · model merging

- [An Empirical Study of Pre-trained Model Selection for Out-of-Distribution Generalization and Calibration](https://hiroki11x.github.io/publications/pre-trained-model-selection-for-ood-generalization-and-calibration/) — TMLR 2025 (2025); Generalization and Foundation Models
- [Generalization Measures under Controlled Covariate Shift: A Regime-Aware Benchmark](https://hiroki11x.github.io/publications/generalization-measures-beyond-iid/) — TMLR 2026 (2026); Generalization and Foundation Models
- [Empirical Study on Optimizer Selection for Out-of-Distribution Generalizations](https://hiroki11x.github.io/publications/optimizer-selection-for-ood-generalization/) — TMLR 2023 (2023); Optimization for Machine Learning; Generalization and Foundation Models