Google DeepMind introduced Decoupled DiLoCo, a distributed training architecture that enables large language models to train across distant data centers with lower bandwidth and greater hardware resiliency. The approach divides training runs across decoupled "islands" of compute with asynchronous data flow, isolating local disruptions so other parts continue learning efficiently. The method builds on prior work called Pathways and avoids communication delays that limited earlier distributed techniques at global scale.
No score is assigned. Sources and their independence are shown in the citation chain below.