bluorion-com/ZClip
Official implementation of the paper: "ZClip: Adaptive Spike Mitigation for LLM Pre-Training".
2 Releases
Latest: 11mo ago
v1.0.0Latest
📋 What's Changed
- Update comments by @akanyaani in https://github.com/bluorion-com/ZClip/pull/1
- Make package installable via `pip`, support gradient accumulation, and max norm-clip correctly for FSDP by @amorehead in https://github.com/bluorion-com/ZClip/pull/3
- Add tests and continuous integration. by @gueraf in https://github.com/bluorion-com/ZClip/pull/9
- Update cimg/python Docker tag to v3.13 by @renovate in https://github.com/bluorion-com/ZClip/pull/10
- Set up Github actions for pypi publishing. by @gueraf in https://github.com/bluorion-com/ZClip/pull/12
✨ New Contributors
- @akanyaani made their first contribution in https://github.com/bluorion-com/ZClip/pull/1
- @amorehead made their first contribution in https://github.com/bluorion-com/ZClip/pull/3
- @gueraf made their first contribution in https://github.com/bluorion-com/ZClip/pull/9
- @renovate made their first contribution in https://github.com/bluorion-com/ZClip/pull/10
- Full Changelog: https://github.com/bluorion-com/ZClip/commits/v1.0.0
v0.0.1Pre-release
