Distributed Training
Training across multiple devices
Top Papers
TorchTitan: One-stop PyTorch native solution for production ready LLM pretraining
Wanchao Liang, Tianyu Liu, Less Wright et al.
Stretching Each Dollar: Diffusion Training from Scratch on a Micro-Budget
Vikash Sehwag, Xianghao Kong, Jingtao Li et al.
BooW-VTON: Boosting In-the-Wild Virtual Try-On via Mask-Free Pseudo Data Training
Xuanpu Zhang, Dan Song, pengxin zhan et al.
Training on the Benchmark Is Not All You Need
Shiwen Ni, Xiangtao Kong, Chengming Li et al.
CO2: Efficient Distributed Training with Full Communication-Computation Overlap
Weigao Sun, Qin Zhen, Weixuan Sun et al.
Mobile Video Diffusion
Haitam Ben Yahia, Denis Korzhenkov, Ioannis Lelekas et al.
Accelerating Neural Field Training via Soft Mining
Shakiba Kheradmand, Daniel Rebain, Gopal Sharma et al.
Integrated Augmented and Virtual Reality Technologies for Realistic Fire Drill Training
Hosan Kang, Jinseong Yang, Beom-Seok Ko et al.
Bridging Training and Execution via Dynamic Directed Graph-Based Communication in Cooperative Multi-Agent Systems
Zhuohui Zhang, Bin He, Bin Cheng et al.
Poplar: Efficient Scaling of Distributed DNN Training on Heterogeneous GPU Clusters
WenZheng Zhang, Yang Hu, Jing Shi et al.
A Quadratic Synchronization Rule for Distributed Deep Learning
Xinran Gu, Kaifeng Lyu, Sanjeev Arora et al.
Faster and Better 3D Splatting via Group Training
Chengbo Wang, Guozheng Ma, Yizhen Lao et al.
Decoupling Training-Free Guided Diffusion by ADMM
Youyuan Zhang, Zehua Liu, Zenan Li et al.
Everywhere Attack: Attacking Locally and Globally to Boost Targeted Transferability
Hui Zeng, Sanshuai Cui, Biwei Chen et al.
PseudoMapTrainer: Learning Online Mapping without HD Maps
Christian Löwens, Thorben Funke, Jingchao Xie et al.
On-Device Diffusion Transformer Policy for Efficient Robot Manipulation
Yiming Wu, Huan Wang, Zhenghao Chen et al.
STraj: Self-training for Bridging the Cross-Geography Gap in Trajectory Prediction
Zhanwei Zhang, Minghao Chen, Zhihong Gu et al.
Distributed Semantic Segmentation with Efficient Joint Source and Task Decoding
Danish Nazir, Timo Bartels, Jan Piewek et al.
ACCO: Accumulate While You Communicate for Communication-Overlapped Sharded LLM Training
Adel Nabli, Louis Fournier, Pierre ERBACHER et al.
InstaTrain: Adaptive Training via Ultra-Fast Natural Annealing within Dynamical Systems
Chuan Liu, Ruibing Song, Chunshu Wu et al.
COGNATE: Acceleration of Sparse Tensor Programs on Emerging Hardware using Transfer Learning
Chamika Sudusinghe, Gerasimos Gerogiannis, Damitha Lenadora et al.
CroCoDL: Cross-device Collaborative Dataset for Localization
Hermann Blum, Alessandro Mercurio, Joshua O'Reilly et al.
EDiT: A Local-SGD-Based Efficient Distributed Training Method for Large Language Models
Jialiang Cheng, Ning Gao, Yun Yue et al.
HAT Swapping: Virtual Agents as Stand-ins for Absent Human Instructors in Virtual Training
Jingjing Zhang, Binyang Han, Ze Dong et al.
Shared, Replicated, or Separated? A Comparative Study of Virtual Workspace Configurations for Collaborative Hands-On Learning
Juliette Le Meudec, Anastasia Bezerianos, Arnaud Prouzeau
Cross-Reality for Autonomous Mobility in Autistic Individuals: Evaluating At-Home VR Training and In-Situ AR Support in a Field Study
Francesco Vona, Mattia Gianotti, Maria Luisa Lorusso et al.
Can People's Brains Synchronize during Remote AR Collaboration?
Jaehwan You, Myeongul Jung, Kwanguk Kim
Ghidorah: Towards Robust Multi-Scale Information Diffusion Prediction via Test-Time Training
Wenting Zhu, Chaozhuo Li, Litian Zhang et al.
The Indoor-Training Effect: Unexpected Gains from Distribution Shifts in the Transition Function
Serena Bono, Spandan Madan, Ishaan Grover et al.
Complexity of Neural Network Training and ETR: Extensions with Effectively Continuous Functions
Teemu Hankala, Miika Hannula, Juha Kontinen et al.
Heterogeneous Test-Time Training for Multi-Modal Person Re-identification
Zi Wang, Huaibo Huang, Aihua Zheng et al.
High-Fidelity Gradient Inversion in Distributed Learning
Zipeng Ye, Wenjian Luo, Qi Zhou et al.
Communication Efficient Distributed Newton Method over Unreliable Networks
Ming Wen, Chengchang Liu, Yuedong Xu
Don’t Drop Your Samples! Coherence-Aware Training Benefits Conditional Diffusion
Nicolas Dufour, Victor Besnier, Vicky Kalogeiton et al.
MS-DETR: Efficient DETR Training with Mixed Supervision
Chuyang Zhao, Yifan Sun, Wenhao Wang et al.
Enhancing Neural Training via a Correlated Dynamics Model
Jonathan Brokman, Roy Betser, Rotem Turjeman et al.
Flag Aggregator: Scalable Distributed Training under Failures and Augmented Losses using Convex Optimization
Hamidreza Almasi, Harsh Mishra, Balajee Vamanan et al.
OmniBal: Towards Fast Instruction-Tuning for Vision-Language Models via Omniverse Computation Balance
Yongqiang Yao, Jingru Tan, Feizhao Zhang et al.
CTBench: A Library and Benchmark for Certified Training
Yuhao Mao, Stefan Balauca, Martin Vechev
DexScale: Automating Data Scaling for Sim2Real Generalizable Robot Control
Guiliang Liu, Yueci Deng, Runyi Zhao et al.
SMART-PC: Skeletal Model Adaptation for Robust Test-Time Training in Point Clouds
Ali Bahri, Moslem Yazdanpanah, Sahar Dastani Oghani et al.
HALoS: Hierarchical Asynchronous Local SGD over Slow Networks for Geo-Distributed Large Language Model Training
Geon-Woo Kim, Junbo Li, Shashidhar Gandham et al.
IT$^3$: Idempotent Test-Time Training
Nikita Durasov, Assaf Shocher, Doruk Oner et al.
Beyond Low-rank Decomposition: A Shortcut Approach for Efficient On-Device Learning
Le-Trung Nguyen, Aël Quélennec, Van-Tam Nguyen et al.
ATA: Adaptive Task Allocation for Efficient Resource Management in Distributed Machine Learning
Artavazd Maranjyan, El Mehdi Saad, Peter Richtarik et al.
Distributed Conformal Prediction via Message Passing
Haifeng Wen, Hong XING, Osvaldo Simeone
NDOT: Neuronal Dynamics-based Online Training for Spiking Neural Networks
Haiyan Jiang, Giulia De Masi, Huan Xiong et al.
Sparse Cocktail: Every Sparse Pattern Every Sparse Ratio All At Once
Zhangheng Li, Shiwei Liu, Tianlong Chen et al.
Position: Exploring the Robustness of Pipeline-Parallelism-Based Decentralized Training
Lin Lu, Chenxi Dai, Wangcheng Tao et al.
Multiply Robust Estimation for Local Distribution Shifts with Multiple Domains
Steven Wilkins-Reeves, Xu Chen, Qi Ma et al.
Fault Tolerant ML: Efficient Meta-Aggregation and Synchronous Training
Tehila Dahan, Kfir Levy
APT: Adaptive Personalized Training for Diffusion Models with Limited Data
JungWoo Chae, Jiyoon Kim, Jaewoong Choi et al.
Towards a Better Theoretical Understanding of Independent Subnetwork Training
Egor Shulgin, Peter Richtarik
Synchronizing Task Behavior: Aligning Multiple Tasks during Test-Time Training
Wooseong Jeong, Jegyeong Cho, Youngho Yoon et al.
FedEL: Federated Elastic Learning for Heterogeneous Devices
Letian Zhang, Bo Chen, Jieming Bian et al.
From Promise to Practice: Realizing High-performance Decentralized Training
Zesen Wang, Jiaojiao Zhang, Xuyang Wu et al.
DistRL: An Asynchronous Distributed Reinforcement Learning Framework for On-Device Control Agent
Taiyi Wang, Zhihao Wu, Jianheng Liu et al.
Rethinking the Starting Point: Collaborative Pre-Training for Federated Downstream Tasks
Yun-Wei Chu, Dong-Jun Han, Seyyedali Hosseinalipour et al.
Multitarget Device-Free Localization via Cross-Domain Wi-Fi RSS Training Data and Attentional Prior Fusion
Na FAN, Zeyue Tian, Amartansh DUBEY et al.
Harnessing Manycore Processors with Distributed Memory for Accelerated Training of Sparse and Recurrent Models
Jan Finkbeiner, Thomas Gmeinder, Mark Pupilli et al.
TinyTrain: Resource-Aware Task-Adaptive Sparse Training of DNNs at the Data-Scarce Edge
Young Kwon, Rui Li, Stylianos Venieris et al.
DyMO: Training-Free Diffusion Model Alignment with Dynamic Multi-Objective Scheduling
Xin Xie, Dong Gong
HOT: Hadamard-based Optimized Training
Seonggon Kim, Juncheol Shin, Seung-taek Woo et al.
TREAD: Token Routing for Efficient Architecture-agnostic Diffusion Training
Felix Krause, Timy Phan, Ming Gui et al.
CoStoDet-DDPM: Collaborative Training of Stochastic and Deterministic Models Improves Surgical Workflow Anticipation and Recognition
Kaixiang Yang, Xin Li, Qiang Li et al.
Mixtures of Subspaces for Bandwidth Efficient Context Parallel Training
Sameera Ramasinghe, Thalaiyasingam Ajanthan, Hadi Mohaghegh Dolatabadi et al.
Analog In-memory Training on General Non-ideal Resistive Elements: The Impact of Response Functions
Zhaoxian Wu, Quan Xiao, Tayfun Gokmen et al.
MeCeFO: Enhancing LLM Training Robustness via Fault-Tolerant Optimization
Rizhen Hu, Yutong He, Ran Yan et al.
AC-DiT: Adaptive Coordination Diffusion Transformer for Mobile Manipulation
Sixiang Chen, Jiaming Liu, Siyuan Qian et al.
Unveiling the Power of Multiple Gossip Steps: A Stability-Based Generalization Analysis in Decentralized Training
DUO: No Compromise to Accuracy Degradation
Jinda Jia, Cong Xie, Hanlin Lu et al.
Accelerating neural network training: An analysis of the AlgoPerf competition
Priya Kasimbeg, Frank Schneider, Runa Eschenhagen et al.
NExUME: Adaptive Training and Inference for DNNs under Intermittent Power Environments
Cyan Subhra Mishra, Deeksha Chaudhary, Jack Sampson et al.
An Asynchronous Bundle Method for Distributed Learning Problems
Daniel Cederberg, Xuyang Wu, Stephen Boyd et al.
Learning to Help in Multi-Class Settings
Yu Wu, Yansong Li, Zeyu Dong et al.