resilient training

ResiHP: Taming LLM Training Failures with Dynamic Hybrid Parallelism

Hybrid parallelism underpins large-scale LLM training across tens of thousands of GPUs. At such scale, hardware failures on individual …

Tenghui Ma, Jihu Guo, Wei Gao, Sitian Lu, Zhisheng YE, Dahua Lin, Hanjing Wang