Megatron-DeepSpeed
deepspeedai/Megatron-DeepSpeed
Ongoing research training transformer language models at scale, including: BERT & GPT-2
2,261stars
Forks
368
Open issues
162
Watchers
2,261
Size
8.1 MB
PythonOther
Created: Jun 21, 2021
Updated: Aug 25, 2026
Last push: Aug 14, 2025
Fork