[r/LocalLLaMA] 1Covenant/Covenant-72B: Largest model so far to be trained on decentralized permissionless GPU nodes

Summary

Covenant AI has introduced Covenant-72B, marking it as the largest model to date trained on decentralized, permissionless GPU nodes. This achievement was made possible by their innovative SparseLoco method, which builds on DiLoCo. SparseLoco significantly reduces communication overhead and synchronization frequency, utilizing a local AdamW optimizer and aggressive top-K sparsification to overcome bandwidth bottlenecks inherent in distributed training.

Continue Reading

Explore related coverage about community news and adjacent AI developments: [r/ML] [D] MYTHOS-INVERSION STRUCTURAL AUDIT, [r/LocalLLaMA] karpathy / autoresearch, [r/ML] [R] Agentic AI and Occupational Displacement: A Multi-Regional Task Exposure Analysis (236 occupations, 5 US metros), [r/ML] Building behavioural response models of public figures using Brain scan data (Predict their next move using psychological modelling) [P].

Related Articles

• [r/ML] [D] MYTHOS-INVERSION STRUCTURAL AUDIT

  March 29, 2026

• [r/LocalLLaMA] karpathy / autoresearch

  March 10, 2026

• [r/ML] [R] Agentic AI and Occupational Displacement: A Multi-Regional Task Exposure Analysis (236 occupations, 5 US metros)

  April 7, 2026

• [r/ML] Building behavioural response models of public figures using Brain scan data (Predict their next move using psychological modelling) [P]

  April 5, 2026