#163 · Primary category： Deep Learning Frameworks
higgsfield
Fault-tolerant, highly scalable GPU orchestration, and a machine learning framework designed for training models with billions to trillions of parameters
Project last updated：05/25/24
GitHub Stars
4.1K
Forks
708
Contributors
4
License
Apache-2.0
Why we included this project
Training a large language model involves a lot of moving parts: multi-node setup, launch scripts, node failures. Higgsfield takes that orchestration layer and replaces it with a Python decorator and a model constructor like Llama70b, then handles node allocation, ZeRO-3 and fully sharded data parallel sharding, and the experiment queue for you. If your team already lives in GitHub, the CI-based flow is a nice fit: pushing code can kick off the training run and surface its UI right in the repo. It's aimed at engineers who want a higher-level distributed training path without giving up PyTorch and DeepSpeed concepts they already know. For anyone running a small GPU fleet and tired of assembling these pieces by hand, it's worth a look as a lighter-weight alternative.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
tensorflow
An Open Source Machine Learning Framework for Everyone
pytorch
Tensors and Dynamic neural networks in Python with strong GPU acceleration
keras
Deep Learning for humans
nanoGPT
The simplest, fastest repository for training/finetuning medium-sized GPTs.
ray
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.