Megatron-DeepSpeed
deepspeedai/Megatron-DeepSpeed
Ongoing research training transformer language models at scale, including: BERT & GPT-2
2,265stars
Forks
366
Open issues
162
Watchers
2,265
Size
8.1 MB
PythonOther
Created: Jun 21, 2021
Updated: Oct 9, 2026
Last push: Aug 14, 2025
Fork